Skip to content

Add instrumental-variable LATE example + regression test (#346)#354

Draft
daimon-pymclabs wants to merge 9 commits into
mainfrom
iv-late-example
Draft

Add instrumental-variable LATE example + regression test (#346)#354
daimon-pymclabs wants to merge 9 commits into
mainfrom
iv-late-example

Conversation

@daimon-pymclabs

Copy link
Copy Markdown
Collaborator

Summary

Closes #346 (part of the IV epic #343). Adds a worked instrumental-variable example and a regression test, demonstrating that pathmc already estimates a continuous-outcome IV model with zero API change through the ~~ correlated-residual operator.

The example reproduces the non-compliance / LATE example from Matheus Facure's Causal Inference for the Brave and True and its PyMC counterpart by Juan Orduz: a randomized push-notification campaign where assignment is random but delivery is not, so engaged customers both receive more pushes and buy more.

The IV spec

spec = """
push_delivered  ~ push_assigned      # first stage (relevance)
in_app_purchase ~ push_delivered     # structural eq — instrument omitted (exclusion)
push_delivered ~~ in_app_purchase    # unobserved-confounder endogeneity
"""
model = pathmc.model(spec, data=df); model.fit()

The ~~ block compiles the two equations into a joint MvNormal with an LKJCholeskyCov residual covariance — the Bayesian limited-information IV structure. The residual correlation soaks up the unobserved engagement confounder; the structural coefficient on push_delivered recovers the LATE.

What the example covers

  • What an IV is — relevance, exclusion, exogeneity — and why those last two are untestable assumptions.
  • Why it's needed here — non-compliance makes treatment-received endogenous; naive OLS is biased ~2× upward.
  • How to implement it in pathmc — the spec mapping, model.graph() / equations(), the PyMC plate, recovery vs. the Wald estimator, the residual-correlation diagnostic, and a do() counterfactual.
  • An honest note that is_identifiable is backdoor-only today (false negative for IV), pointing at IV-aware identification: instrument detection & validation #344.

On simulated data with a known true effect, the IV posterior brackets the truth and the frequentist Wald estimate while naive OLS sits far outside the interval.

Regression test

tests/test_instrumental_variable.py — fixed-seed confounded system with a valid instrument; asserts naive OLS is biased, the IV posterior recovers the true effect, beats naive OLS, and yields a positive endogeneity correlation. Robust at the suite's fast-sampling cap.

🤖 Generated with Claude Code

Reproduces the non-compliance / LATE example from Facure's causality
handbook and its PyMC counterpart, showing that pathmc estimates a
continuous-outcome IV model with zero API change via the ~~
correlated-residual operator:

    push_delivered  ~ push_assigned     # first stage (relevance)
    in_app_purchase ~ push_delivered    # structural eq (instrument excluded)
    push_delivered ~~ in_app_purchase   # unobserved-confounder endogeneity

The worked example explains what an instrumental variable is, why it is
needed under non-compliance, and how the spec maps to the IV assumptions.
On simulated data with a known true effect, naive OLS is biased ~2x upward
while the IV posterior recovers the truth. A fixed-seed regression test
asserts recovery and that the IV estimate beats naive OLS.

Part of #343.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@juanitorduz juanitorduz self-assigned this Jun 26, 2026
@juanitorduz
juanitorduz marked this pull request as draft June 26, 2026 09:59
@juanitorduz juanitorduz added the documentation Improvements or additions to documentation label Jun 26, 2026
daimon-pymclabs and others added 8 commits June 26, 2026 10:03
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The MvNormal residual block coupled variables on very different scales
(purchases ~25 vs delivery ~0.3, a ~23x ratio), which made NUTS crawl
with deep trees. Rescale the outcome (smaller intercept, confounder, and
noise) so the residual-scale ratio drops to ~3x while preserving the
~2x naive-OLS bias story and clean recovery of the known LATE.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Cut the simulated sample to 2,000 customers and match dynamic_pricing's
500/500 sampler settings so the page fits/renders quickly (the full
10k-row do() posterior-predictive pass dominated freeze time). The
~2x naive-OLS bias and clean LATE recovery are unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The do() posterior-predictive pass dominated render time and is
redundant for a linear structural model — the do(1)-do(0) contrast
equals the push_delivered coefficient already shown. Keep the teaching
point as prose so the page renders quickly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
nutpie on the pinned pymc/pytensor stack produces a non-converged
posterior for this binary-treatment IV block (LATE ~11 with an
absurd HDI). The default PyMC NUTS sampler recovers the known LATE
cleanly (mean 0.55, 94% HDI [0.43, 0.69] vs true 0.6; naive OLS 1.23
sits outside the interval). Drop the nutpie request and commit the
executed _freeze/ cache so the docs build restores cached outputs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Collapse the now-short in_app_purchase assignment to one line (ruff)
and regenerate the freeze so its source hash matches the formatted qmd.
Recovered numbers are unchanged (data and seed are identical).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Continuous-outcome IV: worked example + regression test

2 participants