Improve bidder/DSP ID syncing

[bretg]

There's still a gap in the revenue performance between PBJS and PBS that discourages publishers from adopting PBS more enthusiastically. One of the factors behind that gap may be that most PBS adapters don't do a DSP ID sync -- instead they just sync the exchange ID. So when that exchange calls a DSP, the DSP's ID isn't present and the impression value is lower.

Bidders that submit iframe syncs should be able to at least partly address this ID gap where permissions exist. See issues #1554 and #1971 . The requirement would be that their iframe script still only sets the exchange-level ID with Prebid Server, but they could sync DSP IDs back to their exchange so the lookup could be done on the next auction.

However, a single iframe should not sync more than a few DSPs and once a bidder has established their ID in PBS, the iframe won't get called again for a month. Here's it currently works:

1. /cookie_sync is called, iframes are allowed, and bidderA has an iframe sync
2. The iframe hits the PBS/setuid endpoint to establish bidderA-ID in the uids cookie
3. The iframe could also drop redirect pixels to a few DSPs but those redirects need to go back to the bidder to build a map from bidderA-ID to DSP-ID on their side. DSP sync results cannot come to PBS.
4. The next time /cookie_sync is called, bidderA-ID is already in the uids cookie so PBS will not return another iframe to that client. No more DSP syncs for bidderA.

Possible Solution: Iframe multisync

PBS is expanded to support the idea that a given bidder should be able to drop their iframe pixel once per /cookie_sync request until they (optionally) report being done.

1. Expand the PBS usersync config to support a new usersync mode: multisync.

    usersync:
      cookie-family-name: bidderA
      iframe:
        url: ...
      redirect:
        url: ...
      multisync:
        url: ...
        expirehours: 24     // defaults to 168

2. The uids cookie is expanded to carry this value when true.

        "bidderA": {
            "uid": "KX6NAGF8",
            "expires": "2022-10-17T19:39:40.936758973Z",
            "multisync": {
               "supports": true,
               "inprog": true,
               "expires": "2022-10-17T19:39:40.936758973Z"
             }
        },

3. Define new /cookie_sync parameters:
   • multisync-bidders: an array of biddercodes which are allowed to keep dropping iframes until they report being "done". No default.
   • multisync-limit: how many DSP syncs each multisync bidder is allowed to kick off. Default to 5.
4. If multibid is (a) supported, (b) allowed by pub, and (c) either in-progress or ready to refresh, then /cookie_sync returns with the multisync URL for that bidder, setting the 'inprog' flag to true. A macro should be supported so this script knows how many syncs it's allowed to do.

https://sync.example.com/user_sync?gdpr={{gdpr}}&gdpr_consent={{gdpr_consent}}&us_privacy={{us_privacy}}&limit={{multisync_limit}}&redirect={{redirect_url}}

5. Support a new multisyncdone parameter on /setuid which sets the 'inprog' flag defined in point 2 above. So this bidder is allowed to keep syncing until multisyncdone=true.

Potential Concerns

Adding more usersyncing in this era may not be as useful as in the past:

1. Support for 3rd party cookies seems to be in decline
2. More privacy regulations affect setting IDs
3. Prebid supports more than 30 "global" IDs (e.g. unifiedID). DSP-specific IDs may soon be a thing of the past. (?)

Looking for community feedback.

[bretg]

Got feedback from Brian May:

It has been a while since I've been directly involved in ID-syncing, but when I was, we found that ID lifetimes varied significantly both by DSP and by individual browser: some percentage of DSP IDs lasted only hours, some days, some a couple weeks, a few lasted longer and similarly with browsers. I would be surprised if that wasn't still the case and would suggest in light of it that rather than having a static time out, do syncs periodically to verify cached IDs are valid and maintain stats for various entities so that TTLs for IDs can be predicted re-sync's timed accordingly. My underlying concern is that exchanges will end up sending DSPs a lot of stale IDs which is arguably a worse case than sending not DSP IDs at all.

[bmayd]

Thanks for posting, Bret; figured out the issue with my posting.

[MarkMcEachran]

Comments from the field (me):

• Revenue from buyers still prefers cookie-matched users, rather than new ID vendor IDs.
• TCF, while under scrutiny, is easily supported in sync'ing activity, and is supported widely for such purposes.
• Prebid Server is the only supply path that doesn't offer some assistance in Bidder/DSP ID sync'ing.

[bretg]

Discussed in committee. We don't agree on the requirements yet. There's concern about architecting this as a third option. Bret to go back the drawing board and try to combine the iframe options into one.