Researcher portal to allow researchers to use
The tools we are building as part of phase 1 are capable of supporting our in-house research goals and the work of one or two high-engagement collaborators. In phase 2, we hope to develop our tools to the point where they can support an order of magnitude more collaborations.
These developments fall into three major groups. Firstly, we will work to make the prototype “back end” of our experimental platform - comprising Empirica and the statistical-predictive model and design cycle - publicly available via a web-based Researcher Portal. This “front end” will expose to the world: (a) the current state of experiments and statistical models; (b) a list of high-value future experiments ranked in order of their likely information gain; and (c) a signup and payment mechanism that will allow other research teams to run their own experiments with minimal effort and programming expertise, at the marginal cost of paying their experiment participants. Having run their experiment, the researchers will receive data that they can use for their own purposes (e.g. to write a paper) and the data will also be fed back into our statistical models, iterating the train-test cycle.
On this platform, researchers will be able to quickly and easily make a variety of contributions. By selecting one of the model-ranked experiments, they can both test our existing statistical model and improve it. Alternatively, they can choose to run a different experiment motivated by a different theoretical or practical concern, which will also help test and improve the model. This would act as an alternative "acquisition function" that promotes exploration in the design space, as researchers' intuition may prove to be better at guiding sample selection than the platform's algorithms. Additionally, researchers may have conducted experiments outside the platform, in which case they can donate their data to be included in the larger model and assess the impact of their data on improving out-of-sample predictions. In this phase, we will establish standard data-sharing agreements for identifiable data, such as video recordings of deliberations, to protect data privacy and security. We will also work with our institutional review board to create template protocols and reliance agreements that will assist potential collaborators in explaining our system to their own institutional review boards and outlining the expectations for protecting human subjects.