90/10 Credible Intervals for ROAS

[wanyi11]

Hi Sibyl,

Thank you for sharing your scripts here. It's really helpful!

I would like to get your thoughts on calculating the 90/10 credible intervals for the final ROAS results. Coz we want to calculate a range of ROAS so that we can interpret the ROAS with a margin of error. Do you have any ideas on how to do that? Thank you so much!

[sibylhe]

If you want to handle uncertainty - get the lowest possible value and highest possible value, sample MMM parameters. sm2.sampling(data=model_data2, iter=1000, chains=3) yields 1500 sets of (beta, tau, decay, peak), sample 100 of them (or 1000, or use all of them), pass them to the decompose media contribution function, you shall get 100 mc_df, then 100 ROAS, from which you can get the 10th and 90th percentile.

If you want an estimation of ROAS with a confidence interval, use bootstrap method.
Step 1: same as above. Sample 100 parameter sets -> 100 mc_df -> 100 ROAS.
Step 2: calculate the mean of the 100 ROAS (sample mean).
Step 3: repeat 1 & 2 for 30+ times (Central Limit Theorem) -> 30+ sample means. If the distribution of sample mean is normal distribution (CLT holds), you can calculate the 80% confidence interval; if not (CLT does not hold), use the the 10th and 90th percentile.