Skip to content

Add vignette with case study using new model_compare() - #404

Draft
florence-bockting wants to merge 2 commits into
pred_measurefrom
case-study-model-compare
Draft

florence-bockting wants to merge 2 commits into
pred_measurefrom
case-study-model-compare

Conversation

@florence-bockting

@florence-bockting florence-bockting commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #401

Description

This PR adds a vignette that demonstrate a case study, guiding the user through the Bayesian model comparison workflow using the new predictive measure API and the new model_compare() function.

This vignette completes a set of vignettes for loo version 3, focussing:

  • predictive measures (elpd, rmse, r2, rps, ...)
  • predictive schemes (loo, kfold, test,...)
  • [this one] workflow/case study
  • model comparison with model_compare() (focus on API not workflow)

This PR uses AI assistance.

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.92%. Comparing base (de81c46) to head (22a4981).

Additional details and impacted files
@@              Coverage Diff              @@
##           pred_measure     #404   +/-   ##
=============================================
  Coverage         91.92%   91.92%           
=============================================
  Files                38       38           
  Lines              5068     5068           
=============================================
  Hits               4659     4659           
  Misses              409      409           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions

Copy link
Copy Markdown
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 22a4981 is merged into pred_measure:

  • ✔️loo_function: 1.65s -> 1.64s [-2%, +1.03%]
  • ✔️loo_matrix: 1.59s -> 1.59s [-0.82%, +1.04%]
    Further explanation regarding interpretation and methodology can be found in the documentation.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants