Id Modules: not respecting publisher configuration

[patmmccann]

Type of issue

Bug

Description

https://docs.prebid.org/dev-docs/modules/userId.html

shows publishers can choose where to set storage, but several modules ignore eg Verizon

Prebid.js/modules/connectIdSystem.js

Line 45 in 015f12b

function storeObject(obj) {

Lotame appears to not have this setting available and just sets both

Prebid.js/modules/lotamePanoramaIdSystem.js

Line 45 in 015f12b

storage.setCookie(

LiveRamp appears to not use the parameter for html5 or cookie but instead has an additional proprietary parameter that seems related

Prebid.js/modules/identityLinkIdSystem.js

Line 122 in 015f12b

setEnvelopeSource(false);

Feature to add: allow a new publisher configuration option which prevents submodules from writing to storage, so that only userid is responsible for writing and persisting the identifier coming back from the submodule.

[dgirardi]

These are ID modules that use storage directly (grepping for getStorageManager). It doesn't necessarily mean that they don't respect publisher config, but in theory ID modules do not need to deal with storage - it's done for them by the base ID module.

        admixerIdSystem.js  
        adqueryIdSystem.js  
        adriverIdSystem.js  
        amxIdSystem.js  
        connectIdSystem.js  
        criteoIdSystem.js  
        czechAdIdSystem.js  
        dacIdSystem.js  
        deepintentDpesIdSystem.js  
        euidIdSystem.js  
        freepassIdSystem.js  
        ftrackIdSystem.js  
        gravitoIdSystem.js  
        growthCodeIdSystem.js  
        hadronIdSystem.js  
        id5IdSystem.js  
        identityLinkIdSystem.js  
        idxIdSystem.js  
        imuIdSystem.js  
        intentIqIdSystem.js  
        liveIntentIdSystem.js  
        lotamePanoramaIdSystem.js  
        merkleIdSystem.js  
        mwOpenLinkIdSystem.js  
        naveggIdSystem.js  
        novatiqIdSystem.js  
        pairIdSystem.js  
        parrableIdSystem.js  
        publinkIdSystem.js  
        quantcastIdSystem.js  
        teadsIdSystem.js  
        uid2IdSystem.js  
        zeotapIdPlusIdSystem.js  

[patmmccann]

We can see id modules, as noted in the example above, violating the publisher parameter choices for storage.type listed at https://docs.prebid.org/dev-docs/modules/userId.html#basic-configuration

For example, panoramasets the cookie when the publisher chooses html5. This means it is operating outside of how it is configured and documented to behave and is a bug we need to fix.

[patmmccann]

I propose we boot all id modules doing this in Prebid 9. @jdwieland8282 let's talk about it in committee?

[danielsao]

@patmmccann LiveRamp user id module respects the publisher settings in storage.type. The line of code you are referring to is about logging the identity envelope source type (true or false) in a cookie for analytics purposes and not about overwriting the storage location where the actual identity envelope is saved.

[leonardlabat]

Hello @patmmccann

However, Criteo team can you also have your bidder module do the same when it sets cto_bundle?

Do you have an example showing how the storage configuration can be retrieved from a bidder adapter?

[patmmccann]

Might be too ambitious for prebid 9? what do you think @lcorrigall @dgirardi

[patmmccann]

Do you have an example showing how the storage configuration can be retrieved from a bidder adapter?

You could take it as a bidder param potentially?

[jdwieland8282]

I don't think we should boot the modules who are out of compliance. If we include it in pb 9 what does that practically mean?

[muuki88]

If being removed in pb9 , publishers that use those id modules would not upgrade, until the adapter has been resubmitted.

I'm actually in favor of removing ( booting? ) the modules that do not comply. We had issues with cookie headers being too large and when the storage settings are ignored, this may cause frustration.

[jdwieland8282]

Thats a fair point @muuki88

@dgirardi I see your list above, how can I figure out which of those do not respect the pubs cookies|html5 choice?

[patmmccann]

Committee discussion result: we should subset the above list to those that are writing outside of the configured location, not reading outside of configuration.

[patmmccann]

Only thing to do for 9 will be to add doc warnings for adapters using set methods from storage manager.

"This adapter writes to device storage and may not follow the configuration set by the publisher. A future version of Prebid may disallow this behavior."

We also need to compile the actual list. As @danielsao points out, their write is of a different value, not the id returned.

[jdwieland8282]

Let me know how I can help compiling the list.

[patmmccann]

Checking the above list for writes:

Prebid.js/modules/adqueryIdSystem.js

Line 114 in dbf9817

storage.setDataInLocalStorage('qid', myQid);

Prebid.js/modules/adriverIdSystem.js

Line 73 in dbf9817

storage.setCookie('adrcid_cd', new Date().getTime(), now.toUTCString(), 'Lax');

Prebid.js/modules/connectIdSystem.js

Line 58 in dbf9817

storage.setDataInLocalStorage(MODULE_NAME, JSON.stringify(obj));

Prebid.js/modules/dacIdSystem.js

Line 94 in dbf9817

setAoneidToCookie(ret.uid);

Prebid.js/modules/hadronIdSystem.js

Line 104 in dbf9817

storage.setDataInLocalStorage(HADRONID_LOCAL_NAME, hadronId);

Prebid.js/modules/id5IdSystem.js

Line 433 in dbf9817

storeInLocalStorage(ID5_PRIVACY_STORAGE_NAME, JSON.stringify(fetchCallResponse.privacy), NB_EXP_DAYS);

Prebid.js/modules/imuIdSystem.js

Line 71 in dbf9817

storage.setDataInLocalStorage(storageKey, jsonResponse.uid);

Prebid.js/modules/lotamePanoramaIdSystem.js

Line 52 in dbf9817

storage.setCookie(

Prebid.js/modules/teadsIdSystem.js

Line 79 in dbf9817

storage.setCookie(FP_TEADS_ID_COOKIE_NAME, bodyResponse, expirationCookieDate);

Prebid.js/modules/merkleIdSystem.js

Line 41 in dbf9817

storage.setCookie(name, value, expTime.toUTCString(), 'Lax');

Prebid.js/modules/mwOpenLinkIdSystem.js

Line 106 in dbf9817

writeCookie(newMwOlId);

Prebid.js/modules/naveggIdSystem.js

Line 33 in dbf9817

writeCookie(NAVEGG_ID, responseObj[NAVEGG_ID]);

Prebid.js/modules/quantcastIdSystem.js

Line 52 in dbf9817

storage.setCookie(QUANTCAST_FPA, fpa, expires, '/', domain, null);

Intentiq is unique, they always write to local storage, but check for cookie config. If the pub prefers only cookies, they should honor that:
see both

Prebid.js/modules/intentIqIdSystem.js

Line 74 in dbf9817

storage.setDataInLocalStorage(key, value);

and

Prebid.js/modules/intentIqIdSystem.js

Line 128 in dbf9817

const cookieStorageEnabled = typeof configParams.enableCookieStorage === 'boolean' ? configParams.enableCookieStorage : false

As @danielsao points out, Liveramp is not respecting publisher choices on other things they are writing to storage, not the id

Prebid.js/modules/identityLinkIdSystem.js

Line 141 in dbf9817

storage.setCookie('_lr_retry_request', 'true', now.toUTCString());

These appear to be bugs, as 'both' is now an option

Prebid.js/modules/uid2IdSystem_shared.js

Line 685 in dbf9817

const preferLocalStorage = (config.storage !== 'cookie');

Prebid.js/modules/ftrackIdSystem.js

Line 178 in dbf9817

utils.logWarn(LOG_PREFIX + 'config.storage.type recommended to be "' + LOCAL_STORAGE + '".');

[jdwieland8282]

How does this text sound for our docs repo?

Please be aware this UserId sub adapter may not respect `storage.type` parameter setting. Please reach out directly to the sub adapter maintainer to ensure your storage location selection is respected.

[jdwieland8282]

@dgirardi can you find a module that respects the param.storage.type so we id maintainers can see how this should be supported?

[patmmccann]

@dgirardi can you find a module that respects the param.storage.type so we id maintainers can see how this should be supported?

Yandex recently simplified their submission to conform

See
#11196 (comment)

[abazylewicz-id5]

Is it ok if in our user module, when we do a server call to get an ID, in the response we set cookie with given ID? Or we would need to require cookie storage/not set the cookie when our module is configured to use only html5 storage type?

If we have an option to use external module through explicit module configuration parameter, does the external module also should respect prebid storage configuration or it's enough if we add to the description that the external module uses some additional storage regardless of what's set in prebid storage type?

(Asking here instead of separate issue as it's related and may help others)

[patmmccann]

Your response set cookie header is no problem. The problem is if your js sets a cookie in the first party context when the publisher directed you to use local storage for the first party. For the external module, why would you want to ignore your customers instructions? it's not very friendly to them. We can't police the external js activity, but I do not recommend you set first party cookies when pubs ask you not to. They may have good reasons, respect your customers.

[abazylewicz-id5]

Thanks @patmmccann , for some reason I missed the fact that Prebid indeed can set first party cookies. We do not store any first party cookies in external module, and submitted the fix to the storing on prebid side in this MR: #11965

[jdwieland8282]

I heard offline from Mediawalla and Lotame that they have fixed their modules, can anyone confirm?

[jdwieland8282]

Mediawallah's fix #12149

[Tonsil]

I heard offline from Mediawalla and Lotame that they have fixed their modules, can anyone confirm?

As a Lotame representative, I can tell you that work has been prioritized and planned for Q4 but not yet completed. Is there a deadline for when this change needs to go live? I tried to find a schedule posted for "10.0 API Change" timing, but was unsuccessful.