Proposal: <podcast:events> tag, enabling podcast publishers to receive unforgeable inbound events from other entities in the open podcast ecosystem

[johnspurlock]

I've been listening and thinking about several conversations in the podcast world recently, and I think it would great if we could come up with a solid way of letting podcasters receive inbound events from verifiable 3rd party actors (podcast apps, directories, other hosts) with useful information for them. For example, a way for trusted podcast apps to send unforgeable information about subscribers/play information back to the show.

Ideally, we could do this in a way similar to the other podcast namespace tags, that kept the podcast feed publisher in control, and as the source of truth, while keeping things as simple as possible and without needing preauthorization or shared secrets, or certifications, etc.

Instead of a wall of markdown here in the github issue, I've started a quick Google doc with the general overview of a system I think could work well, and am publishing it publicly to see what everyone thinks and get the conversation started:

[Google Doc] Proposal: <podcast:events> tag, enabling podcast publishers to receive unforgeable inbound events from other entities in the open podcast ecosystem

It's meant to be readable by anyone, so I tried to avoid rabbit-holing on a bunch of detailed example code, but there should be enough in there for folks to get the idea.

2022-10-06: Published a new skymethod/podcast-events GitHub repo with a full implementation of both the sending side and the receiving side. Also a validator to test client implementations at: https://podcast-events-validator.op3.dev/. Both linked in the proposal doc

2022-10-22: Added a new "Event batching" section about how to reduce the number of calls required to the same receiver (e.g. the same podcast hosting company) by sending events for multiple feeds in the same payload

[theDanielJLewis]

I foresee some feet-dragging for app-developers to support this, but I do really like the idea and see a ton of potential for it!

I have a few thoughts for potential improvement.

1. I suggest making it <podcast:webhook> instead of <podcast:events>. I think "webhook" communicates its purpose better, avoids potential confusion, and leaves the "events" word usable for other plain-English uses (like a way for a podcaster to indicate layman "events," like a meetup, a live-stream, etc.). And before anyone says, "It doesn't matter what we call it, and only developers will see it," I think it does matter what we call it because podcasters need to be able to easily understand it, ask their publishing tools about it, and for it to be easy to educate them.
2. Your example showed this in <channel>. I think there could be some uses for putting it in <item> tags, too.
3. For any kind of episode event, I think we should send the item GUID instead of the enclosure URL because the GUID is supposed to never change for an episode, but enclosure URLs can change at any time.

I could see such a webhook receiving all kinds of information (not necessarily that these are the best ways to send such information, but this is only for brainstorming), whenever:

• someone follows or unfollows the show,
• someone comments on an episode,
• someone shares an episode,
• someone deletes an episode without finishing it,
• the podcast is featured in the app catalog,
• the podcast receives a rating or review,
• the podcast's catalog listing is visited from an in-app ad (paired with when someone follows a show, this can provide great conversion data!)

[jamescridland]

This seems quite close in functionality to NPR's RAD proposal, and I wonder whether it's worth reviewing that for ideas and gotchas of how it was proposed there. Particularly, the opportunity to request the app to "tell me if this point was played" was helpful for program research and potentially for ad delivery.

Improvements on it are the use of signed data, and as @theDanielJLewis says, it could be extensible with a number of different ideas.

I like the webhook name suggestion. Like switching text to txt, I think it's important to use unambiguous language that helps a rookie get up to speed as fast as possible.

I would also suggest it's unlikely to get take-up in apps: unless a) someone pays them to implement it; or b) someone associated with this proposal builds an app that we can get behind

[johnspurlock]

Some differences between this and RAD (spec):

• RAD is primarily designed to report advertising events (see their event properties), this proposal is primarily designed to raise the level of quality listen metrics available to publishers without taking a step back on listener privacy in any way
• It is not helpful to advertisers directly, only in the aggregate sense that publishers will have a better idea how many of their downloads were actually listens, for participating podcast apps
• RAD events can be easily forged, the 'listen' events in this proposal cannot
• RAD events require embedding information into ID3 tags (!), this proposal does not

More context around this proposal: there has been some indications by some of the large podcast apps that they would be willing to send this listen data (which they already collect) securely to publishers in order to help raise the quality bar and prevent overbilling, but will only do it if it does not take a step back on privacy in any way.

The mechanism this proposal uses was specifically chosen to be something that Apple, for example, would be willing to implement - once a consensus is reached on the payload. The auth mechanism (JWT with public JWK url sets) in particular was chosen since it's not only an industry standard these days, but also used by Apple in other callback apis where security is key (Sign-in with Apple / App Store In-app purchase callback notifications), ie their security team has already blessed something similar.

[johnspurlock]

Re: embedding in apps, this proposal is strictly server-to-server, since each request needs to be signed with the private side of the RSA keypair. The private side would never leave the sender's servers or be embedded in a client app or website, etc.

The flow would be that the app sends listen data down to their servers in the same way they do today, then periodically send it server-to-server for feeds that declare <podcast:events>

[daveajones]

After reading through everything, I wonder would this be doable as podping notifications? Same payload, but without the need for JWT since Hive does the authentication and it requires staking. Perhaps you are worried about message volume overwhelming Hive?

[theDanielJLewis]

Bringing up the difference with RAD inspired a thought. This kind of webhook indicator would receive anything the app sends it. But should we consider a way for podcasts to indicate (probably in the episode JSON data) particular triggers? Like ping the webhook if X chapter is completed, or if XX:XX point is played, or if X% is played.

[johnspurlock]

I wonder would this be doable as podping notifications?

The thing is I think we want something that the app sends securely, and targeted only to the show, and the show's declared third parties. Hive would leave the listen data wide open, right?

I guess there would be nothing preventing someone from starting a service that implemented the receiving side of this protocol (something DJ could do in a weekend!), got used by podcasters in their feeds, and turned around and republished that data back out to Hive or anywhere else - but it's the podcaster's choice in this case.

[jamescridland]

Listening to the board meeting with Dave and Adam.

I like the idea of a different example rather than starting with "listens", given the almost instant "but NPR RAD, blah blah" reaction.

One that seems to be a great and simple usecase is "reviews". When I get a review on a podcast in Podverse, send that review back to me using a webhook.

I can absolutely build a receiver for Podnews (it would be super dumb, just capturing the info in a database) if you need a podcast using one.

[johnspurlock]

One that seems to be a great and simple usecase is "reviews"

I like this idea! I'll try to compile a quick summary of all of the great ideas from folks I've heard so far and put them in the doc, and provide ratings payload examples alongside listen info so it doesn't seem so tied to one use case.

Strawman ratings item-level payload(s):

Like all event payloads, fields common to all events can be included once in the top-level object instead of repeated in every item-level payload.

All fields are required except the ones marked optional.

Rating summary

Could be sent daily/weekly if the app doesn't want to send every rating/review.

{
  "kind": "rating-summary",
  "asof": "2022-10-08T16:00:00.000Z",
  "rating": 4.7,
  "ratingScaleMin": 1,
  "ratingScaleMax": 5,
  "count": 955,
  "episodeUrl": "<episode url that was downloaded>", // optional, only for apps that support episode-level reviews
  "feedUrl": "<feed of the subject show in question>",
  "userAgent": "<user agent that downloaded the episode>",
  "referer": "<referer that downloaded the episode>", // optional, only applicable for webapps
}

Rating

For apps willing to send every rating/review.

{
  "kind": "rating",
  "time": "2022-10-08T16:49:00.226Z",
  "rating": 4,
  "ratingScaleMin": 1,
  "ratingScaleMax": 5,
  "review": "Hats off to Jane Doe who excels in character development and tells a great story with brilliant historical context.",
  "author", "JohnDoe",
  "episodeUrl": "<episode url that was downloaded>", // optional, only for apps that support episode-level reviews
  "feedUrl": "<feed of the subject show in question>",
  "userAgent": "<user agent that downloaded the episode>",
  "referer": "<referer that downloaded the episode>", // optional, only applicable for webapps
}

Update: I put these in the google doc

[theDanielJLewis]

Consider what could be done with a tool like Zapier or IFTTT. Both services can offer webhooks for listening. So doing something in the podcast app could trigger a Zapier or IFTTT action.

One (not necessarily recommended) example could be a system to auto-tweet whenever my show gets a new follower, and then promotes my follow page. "Someone from Cincinnati just subscribed to The Audacity to Podcast. Have you? [URL]"

@johnspurlock, I have a ton of thoughts of how to best format rating and review data (since that is literally my business), but we can discuss that later.

[johnspurlock]

The key thing about this proposal is that the requests are signed with strong encryption and need to be verified by the receiver before being processed or forwarded on. It's a way to setup an unforgeable/unspoofable channel from the podcast app -> feed owner (podcaster/podcast hosting company).

So as long as first receiver (the inboxUrl declared in the tag) can do the encryption necessary to verify the signature of the payloads coming in, it would be ok.

I'm thinking that the flow would be that the podcast hosting company (or 3rd party) declared as the inboxUrl in the tag would be the first receiver, trusted to properly verify the events and handle policy around which apps are trusted. Then provide checkboxes/fields in their podcaster-facing CMS for various event hooks like: "rating webhook url", which could then be ifttt/zapier/webhook-co-of-the-month

[theDanielJLewis]

Perhaps there could be a public key, or some simple authentication process, that podcasters could give to various apps, or else there's a central list of approved apps allowed to send payloads to the webhook URL.

For example, Libsyn would already know have the signature stuff in place to authenticate anything from Podverse, Overcast, Apple Podcasts, and such. But for any fringe app or service, the podcaster needs to get a public API key from their hosting provider (or Zapier) that they provide to that app in order to give them permission to send the data. Anything gets block that comes in that's not preapproved by the webhook provider or using a key the podcaster has authorized.