Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BaseAdapter for the Prebid 0.x -> 1.x transition #1494

Merged
merged 22 commits into from
Sep 13, 2017
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
c1080dd
Added a base adapter for single-request adapters, and ported the appn…
dbemiller Aug 17, 2017
70fcf51
Renamed the SingleRequestBidder to BidderFactory. Updated it to handl…
dbemiller Aug 18, 2017
211e7fc
Added a unit test for the delayExecution function.
dbemiller Aug 18, 2017
6246eaf
Merged from master. Fixed conflicts.
dbemiller Aug 18, 2017
d509150
Made newBidder a default import. Added some unit tests.
dbemiller Aug 18, 2017
8d4ebc1
Added more tests.
dbemiller Aug 21, 2017
7ffc16f
Merge branch 'master' of https://github.com/prebid/Prebid.js into sin…
dbemiller Aug 21, 2017
1f23044
Added more tests, and fixed a few bugs.
dbemiller Aug 22, 2017
8d1efa6
Merged from master. Fixed a conflict.
dbemiller Aug 23, 2017
a627a43
Changed an error to a log message. Fixed a small bug.
dbemiller Aug 23, 2017
ff99ff6
Merged from master, and fixed conflicts.
dbemiller Aug 31, 2017
c246aa6
Did the no-brainer improvements from PR comments.
dbemiller Aug 31, 2017
6c80c3d
Added spec-level support for aliases and mediaTypes. Aliases may stil…
dbemiller Aug 31, 2017
37399c7
Added support for aliases. Added more tests
dbemiller Sep 1, 2017
1b42995
Cleaned up some unnecessary code.
dbemiller Sep 1, 2017
5c3d645
Removed the GET/POST constants. Fixed some typos, and renamed some Re…
dbemiller Sep 1, 2017
31899f4
Merged from master. Fixed conflicts.
dbemiller Sep 6, 2017
735f2bc
Re-added some code for outstream rendering, which was apparently lost…
dbemiller Sep 6, 2017
dd4e595
Removed confusing use of this
dbemiller Sep 6, 2017
25f822a
Fixed lint error
dbemiller Sep 6, 2017
e63f2d6
Moved JSON parsing into the bidderFactory, and moved the JSDocs to th…
dbemiller Sep 6, 2017
13d3911
Removed placementCode from everywhere I could.
dbemiller Sep 7, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
606 changes: 285 additions & 321 deletions modules/appnexusAstBidAdapter.js

Large diffs are not rendered by default.

8 changes: 8 additions & 0 deletions src/Renderer.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,14 @@
import { loadScript } from './adloader';
import * as utils from './utils';

/**
* @typedef {object} Renderer
*
* A Renderer stores some functions which are used to render a particular Bid.
* These are used in Outstream Video Bids, returned on the Bid by the adapter, and will
* be used to render that bid unless the Publisher overrides them.
*/

export function Renderer(options) {
const { url, config, id, callback, loaded } = options;
this.url = url;
Expand Down
240 changes: 240 additions & 0 deletions src/adapters/bidderFactory.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,240 @@
import Adapter from 'src/adapter';
import bidmanager from 'src/bidmanager';
import bidfactory from 'src/bidfactory';
import { STATUS } from 'src/constants';

import { ajax, GET, POST } from 'src/ajax';
import { logWarn, logError, parseQueryStringParameters, delayExecution } from 'src/utils';

/**
* This file aims to support Adapters during the Prebid 0.x -> 1.x transition.
*
* Prebid 1.x and Prebid 0.x will be in separate branches--perhaps for a long time.
* This function defines an API for adapter construction which is compatible with both versions.
* Adapters which use it can maintain their code in master, and only this file will need to change
* in the 1.x branch.
*
* Typical usage looks something like:
*
* const adapter = newBidder({
* code: 'myBidderCode',
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice and easy for implementers if we allowed aliases to be defined here somehow

e.g.

const adapter = newBidder({
  code: 'rubicon',
  aliases: ['rubiconLite', 'roooobicon']

* areParamsValid: function(paramsObject) { return true/false },
* buildRequests: function(bidRequests) { return some ServerRequest(s) },
* interpretResponse: function(oneServerResponse) { return some Bids, or throw an error. }
* });
*/

/**
* This function aims to minimize the Adapter-specific code for "single request" Bidders. That is, if your adapter
* only makes a single request to the server for bids, regardless of how many bid requests or ad units are in
* the Auction, then this function is for you.
*
* @param {BidderSpec} spec An object containing the bare-bones functions we need to make a Bidder.
*/
export default function newBidder(spec) {
return Object.assign(new Adapter(spec.code), {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to figure out how aliasing is going to work with this new pattern. Currently I have it calling new on the .constructor link of an existing instance. In this case it would be an instance of just plain Adapter, which means it wouldn't work.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added support for these. I'm not sure this is the best way, though... so let me know if you have better ideas.

If we think this API is general enough to be the only base class, I might re-organize the code a bit so that "bidder creation" is in a separate file from "bidder registration".

I still haven't heard anyone weigh in on whether or not this handles all the 1.0 use-cases, though, or whether i'm overly-constraining adapters with it.

callBids: function(bidsRequest) {
if (!bidsRequest.bids || !bidsRequest.bids.filter) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like an array check. Would be more clear as if (!Array.isArray(bidsRequest.bids) ) {

return;
}

// callBids must add a NO_BID response for _every_ placementCode, in order for the auction to
// end properly. This map stores placement codes which we've made _real_ bids on.
//
// As we add _real_ bids to the bidmanager, we'll log the placementCodes here too. Once all the real
// bids have been added, fillNoBids() can be called to end the auction.
//
// In Prebid 1.0, this will be simplified to use the `addBidResponse` and `done` callbacks.
const placementCodesHandled = {};
function addBidWithCode(placementCode, bid) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are using two names for same thing in our codebase i.e placementCode and adUnitCode. From 1.0 we should just use code or adUnitCode
Gist for adUnit changes https://gist.github.com/mkendall07/51ee5f6b9f2df01a89162cf6de7fe5b6#adunit-obj

Copy link
Contributor Author

@dbemiller dbemiller Sep 7, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did not know that... I removed placementCode from all the comments and local variable names.

I think I still have to use bidRequest.placementCode in Prebid 0.x, though, right? Or is there somewhere else I should be getting it from?

placementCodesHandled[placementCode] = true;
bidmanager.addBidResponse(placementCode, bid);
}
function fillNoBids() {
bidsRequest.bids
.map(bidRequest => bidRequest.placementCode)
.forEach(placementCode => {
if (placementCode && !placementCodesHandled[placementCode]) {
bidmanager.addBidResponse(placementCode, newEmptyBid());
}
});
}

const bidRequests = bidsRequest.bids.filter(filterAndWarn);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These names are confusing, especially here. I think bidRequests is fine, but bidsRequest I think is called bidderRequest in other parts of Prebid.

if (bidRequests.length === 0) {
fillNoBids();
return;
}

let requests = spec.buildRequests(bidRequests);
if (!requests || requests.length === 0) {
fillNoBids();
return;
}
if (!Array.isArray(requests)) {
requests = [requests];
}

// Callbacks don't compose as nicely as Promises. We should call fillNoBids() once _all_ the
// Server requests have returned and been processed. Since `ajax` accepts a single callback,
// we need to rig up a function which only executes after all the requests have been responded.
const onResponse = delayExecution(fillNoBids, requests.length)
requests.forEach(processRequest);

function processRequest(request) {
switch (request.type) {
case GET:
ajax(
`${request.endpoint}?${parseQueryStringParameters(request.data)}`,
{
success: onSuccess,
error: onFailure
},
undefined,
{
method: GET,
withCredentials: true
}
);
break;
case POST:
ajax(
request.endpoint,
{
success: onSuccess,
error: onFailure
},
typeof request.data === 'string' ? request.data : JSON.stringify(request.data),
{
method: POST,
contentType: 'text/plain',
withCredentials: true
}
);
break;
default:
logWarn(`Skipping invalid request from ${spec.code}. Request type ${request.type} must be GET or POST`);
onResponse();
}
}

// If the server responds successfully, use the adapter code to unpack the Bids from it.
// If the adapter code fails, no bids should be added. After all the bids have been added, make
// sure to call the `onResponse` function so that we're one step closer to calling fillNoBids().
function onSuccess(response) {
const bidRequestMap = {};
bidRequests.forEach(bid => {
bidRequestMap[bid.bidId] = bid;
});
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this bidRequestMap recreated for each successful request? Couldn't it just be made once in the same context as the bidRequests?

function addBidUsingRequestMap(bid) {
const bidRequest = bidRequestMap[bid.requestId];
if (bidRequest) {
const prebidBid = Object.assign(bidfactory.createBid(STATUS.GOOD, bidRequest), bid);
addBidWithCode(bidRequest.placementCode, prebidBid);
} else {
logWarn(`Bidder ${spec.code} made bid for unknown request ID: ${bid.requestId}. Ignoring.`);
}
}

let bids;
try {
bids = spec.interpretResponse(response);
} catch (err) {
logError(`Bidder ${spec.code} failed to interpret the server's response. Continuing without bids`, null, err);
onResponse();
Copy link
Collaborator

@snapwich snapwich Aug 31, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the calling of onResponse here is a good use case for a finally clause so you don't have to scatter as many calls around.

I'd wrap all this code in a try block and put the onResponse in a finally so it's obvious that it will always be called.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the goal here... but I'm really trying not to catch unexpected errors either.

For example, while writing this I originally had a bug in my addBidUsingRequestMap function where something was unexpectedly undefined.

I easily may never have caught that if it'd been inside the try block.

return;
}

if (bids) {
if (bids.forEach) {
bids.forEach(addBidUsingRequestMap);
} else {
addBidUsingRequestMap(bids);
}
}
onResponse();
}

// If the server responds with an error, there's not much we can do. Log it, and make sure to
// call onResponse() so that we're one step closer to calling fillNoBids().
function onFailure(err) {
logError(`Server call for ${spec.code} failed: ${err}. Continuing without bids.`);
onResponse();
}
}
});

function filterAndWarn(bid) {
if (!spec.areParamsValid(bid.params)) {
logWarn(`Invalid bid sent to bidder ${spec.code}: ${JSON.stringify(bid)}`);
return false;
}
return true;
}

function newEmptyBid() {
const bid = bidfactory.createBid(STATUS.NO_BID);
bid.code = spec.code;
bid.bidderCode = spec.code;
return bid;
}
}

/**
* @typedef {object} ServerRequest
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these typedefs are awesome and will help with a lot of the confusion that currently exists around what type of objects Prebid.js expects to operate with. Do you you think it'd be worthwhile making constructor functions that are used to create these objects?

e.g.

var serverRequest = new ServerRequest('GET', ENDPOINT, data);

I think this would help with data validation. We probably want to validate that the adapters are giving us back the correct data types. But rather than do stuff like if(!serverRequest.type || !serverRequest.endpoint ... we could just do if(!serverRequest instanceof ServerRequest) and the constructor function would could check for required parameters.

Copy link
Contributor Author

@dbemiller dbemiller Sep 1, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been thinking about this... but I'm not so convinced.

First: instanceof is a pretty flimsy check in javascript. For example:

var serverRequest = new ServerRequest(...);
delete serverRequest.type

serverRequest.type // Is undefined
serverRequest instanceof Request // Is true

Second: it sounds like all the complex validation code has just moved into the constructor. Rather than if (!serverRequest.type), the constructor has to do if (!type). I can see some advantage in having the validation code centralized... but if that's the goal, I'd rather define it an isRequest(obj) function somewhere.

Third: constructors are a bad pattern. To be honest, I don't expect the drawbacks to affect this area much... but I do want to discourage them from spreading around the codebase. There's some good discussion on them in (2) and (3) here


Data validation is an open question as far as I'm concerned, though.

I left it out here because I was thinking of how React totally disabled all runtime property validation in the prod version of their library because it was found to be a notable performance drain. Granted, React apps have hundreds (or even thousands) of nested components... so it may not be a fair role model. Since everyone in adtech seems to be super concerned about performance, though, I figured I'd leave it out.

Worth mentioning: Assuming management schedules time for me to work on it, I have a followup PR I hope to make for "simple, unified adapter tests." Basically I want to make it so that adapters can define some JSON files for valid bid params and expected server responses, and we can centralize all the actual adapter test code. One of those tests would be making sure that the spec only returns request objects with valid params. If we do it at test time, then it doesn't need to be done at runtime.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's true, there are ways to subvert instanceof checks. The reason I like them is because, as you said, type validation is expensive and instanceof checks are not. Using a constructor function to define a type allows you to incur a one-time hit for the validation of an object (and then use instanceof thereafter), as opposed to isRequest(obj) which is a many-time hit for each time that object is used. That's the problem React encountered as they had to validate propTypes each type a component instance was used.

Either way, this isn't a make or break issue for me, it's just something I think of every time I see if (obj.property .... Also, if we did do this we'd have to expose the constructor functions on the public API as well so external adapters could pass the instanceof checks, which isn't terrible but it's more of a hassle than just using typedef.

Copy link
Contributor Author

@dbemiller dbemiller Sep 5, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah... I do understand where you're coming from, but I think I'd prefer to hold off on this for now.

I'd feel better talking about how to optimize runtime validation after people see the tests I have in mind, if they still think they have value.

*
* @param {('GET'|'POST')} type The type of request which this is.
* @param {string} endpoint The endpoint for the request. For example, "//bids.example.com".
* @param {string|object} data Data to be sent in the request.
* If this is a GET request, they'll become query params. If it's a POST request, they'll be added to the body.
* Strings will be added as-is. Objects will be unpacked into query params based on key/value mappings, or
* JSON-serialized into the Request body.
*/

/**
* @typedef {object} BidRequest
*
* @param {string} bidId A string which uniquely identifies this BidRequest in the current Auction.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These should be @property instead of @param for typedefs.

* @param {object} params Any bidder-specific params which the publisher used in their bid request.
* This is guaranteed to have passed the spec.areParamsValid() test.
*/

/**
* @typedef {object} Bid
*
* @param {string} requestId The specific BidRequest which this bid is aimed at.
* This should correspond to one of the
* @param {string} ad A URL which can be used to load this ad, if it's chosen by the publisher.
* @param {number} cpm The bid price, in US cents per thousand impressions.
* @param {number} height The height of the ad, in pixels.
* @param {number} width The width of the ad, in pixels.
*
* @param [Renderer] renderer A Renderer which can be used as a default for this bid,
* if the publisher doesn't override it. This is only relevant for Outstream Video bids.
*/

/**
* @typedef {object} BidderSpec An object containing the adapter-specific functions needed to
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe just me but I like to see these typedefs inline with the code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved these, but I'm uh.... not sure I did what you had in mind.

The first mention of BidderSpec is inside an @param {BidderSpec} spec. That block documents the function... so the closest places for the typedef are either before or after the function implementation.

I moved it before the function implementation... which might be better? I dunno though.

Let me know if you have something better in mind. The BidderSpec is used at the top-level functions... and all the other typedef blocks in here are used inside the BidderSpec.

* make a Bidder.
*
* @property {string} code A code which will be used to uniquely identify this bidder. This should be the same
* one as is used in the call to registerBidAdapter
* @property {function(object): boolean} areParamsValid Determines whether or not the given object has all the params
* needed to make a valid request.
* @property {function(BidRequest[]): ServerRequest|ServerRequest[]} buildRequests Build the request to the Server which
* requests Bids for the given array of Requests. Each BidRequest in the argument array is guaranteed to have
* a "params" property which has passed the areParamsValid() test
* @property {function(*): Bid[]} interpretResponse Given a successful response from the Server, interpret it
* and return the Bid objects. This function will be run inside a try/catch. If it throws any errors, your
* bids will be discarded.
* @property {function(): UserSyncInfo[]} fetchUserSyncs
*/

/**
* TODO: Move this to the UserSync module after that PR is merged.
*
* @typedef {object} UserSyncInfo
*
* @param {('image'|'iframe')} type The type of user sync to be done.
* @param {string} url The URL which makes the sync happen.
*/
7 changes: 5 additions & 2 deletions src/ajax.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@ var utils = require('./utils');
const XHR_DONE = 4;
let _timeout = 3000;

export const GET = 'GET';
export const POST = 'POST';
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need these. I think constants are useful when you're dealing with magic numbers or non-obvious strings, but the strings 'POST' and 'GET' are sufficiently descriptive and this makes an extra dependency for all the adapters.

Copy link
Contributor Author

@dbemiller dbemiller Aug 31, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ironically... I actually added these because I couldn't remember where this ajax function was expecting GET or get. Figured this would save me from having to look it up the next time.

If "10" is "I refuse to use strings", and "1" is "I refuse to use constants", I'm like... a 6. Enough to challenge you here, but not very hard. How strong is your preference?

Also open to other peoples' feedback, if people want to vote.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this were our API, then I'd probably be more open to the idea of adding these constants. But this is just a layer on top of an existing API that's already pretty standardized around using 'GET', 'POST', etc strings: https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest/open

Copy link
Contributor Author

@dbemiller dbemiller Sep 1, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aha... ok, that makes plenty of sense then. My javascript inexperience is showing itself... didn't know those were so well established.

I'll take the constants out. I should probably rename type to method, too... will take a pass for consistency, now that I know where the conventions are coming from.


/**
* Simple IE9+ and cross-browser ajax request function
* Note: x-domain requests in IE9 do not support the use of cookies
Expand All @@ -22,7 +25,7 @@ export function ajax(url, callback, data, options = {}) {
try {
let x;
let useXDomainRequest = false;
let method = options.method || (data ? 'POST' : 'GET');
let method = options.method || (data ? POST : GET);

let callbacks = typeof callback === 'object' ? callback : {
success: function() {
Expand Down Expand Up @@ -97,7 +100,7 @@ export function ajax(url, callback, data, options = {}) {
}
x.setRequestHeader('Content-Type', options.contentType || 'text/plain');
}
x.send(method === 'POST' && data);
x.send(method === POST && data);
} catch (error) {
utils.logError('xhr construction', error);
}
Expand Down
24 changes: 24 additions & 0 deletions src/utils.js
Original file line number Diff line number Diff line change
Expand Up @@ -666,7 +666,31 @@ export function getBidderRequest(bidder, adUnitCode) {
}

/**
* Given a function, return a function which only executes the original after
* it's been called numRequiredCalls times.
*
* Note that the arguments from the previous calls will *not* be forwarded to the original function.
* Only the final call's arguments matter.
*
* @param {function} func The function which should be executed, once the returned function has been executed
* numRequiredCalls times.
* @param {int} numRequiredCalls The number of times which the returned function needs to be called before
* func is.
*/
export function delayExecution(func, numRequiredCalls) {
if (numRequiredCalls < 1) {
throw new Error(`numRequiredCalls must be a positive number. Got ${numRequiredCalls}`);
}
let numCalls = 0;
return function () {
numCalls++;
if (numCalls === numRequiredCalls) {
func.apply(null, arguments);
}
}
}

/**
* https://stackoverflow.com/a/34890276/428704
* @export
* @param {array} xs
Expand Down
6 changes: 1 addition & 5 deletions test/spec/modules/appnexusAstBidAdapter_spec.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { expect } from 'chai';
import Adapter from 'modules/appnexusAstBidAdapter';
import { adapter } from 'modules/appnexusAstBidAdapter';
import bidmanager from 'src/bidmanager';

const ENDPOINT = '//ib.adnxs.com/ut/v3/prebid';
Expand Down Expand Up @@ -61,10 +61,6 @@ const RESPONSE = {
};

describe('AppNexusAdapter', () => {
let adapter;

beforeEach(() => adapter = new Adapter());

describe('request function', () => {
let xhr;
let requests;
Expand Down
Loading