Skip to content

Draft (new feature) : Model to estimate when a intervention had effect #480

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 23 commits into
base: main
Choose a base branch
from

Conversation

JeanVanDyk
Copy link

@JeanVanDyk JeanVanDyk commented May 28, 2025

New Feature: InterventionTimeEstimator for Unknown Treatment Timing

This PR introduces a new model, InterventionTimeEstimator, designed to estimate when an intervention has an effect in a time series — especially in cases where the exact treatment time is unknown.


Use Case

  • Enhances the Interrupted Time Series (ITS) feature by providing a way to infer the likely time of intervention
  • Supports scenarios where the treatment onset is uncertain or delayed
  • Helps identify lagged effects between intervention and observable outcomes

This addition gives users a flexible, Bayesian approach to model treatment timing uncertainty directly within the CausalPy framework.


Notes / Open Questions

  • Where should this model fit into the CausalPy workflow?
    I’m unsure whether InterventionTimeEstimator should be integrated within the InterruptedTimeSeries (ITS) feature, or used as a standalone tool.
    This affects how a user-defined model could be supported.
    Depending on the intended usage, I can propose a solution to allow users to inject their own custom models.

  • Custom model usage — base vs. intervention
    Should users be able to:

    • Provide a custom model to represent the base time series (e.g. intercept, trend, seasonality)?
    • Provide a custom model to capture the intervention effect (e.g. shape or dynamics of the post-switch impact)?
    • Or support both?
  • Covariates
    I considered adding time-varying covariates to improve the fit. Would that be useful or out of scope?

  • Multivariate Time Series
    It's relatively easy to extend the model for multivariate input. Let me know if this is something you'd like to see.


Model Summary

  • Inputs:

    • t: 1D array of time points
    • y: 1D array of observed values
    • Optional span: restricts the window for switchpoint detection
    • Optional coords: can include seasons for modeling periodic effects
    • effect: list of components, e.g. "trend", "level", "impulse"
    • grain_season: number of time steps per season
  • Model Components:

    • Time series is modeled as:
      intercept + trend + seasonal
    • A Uniform prior is used to place a switchpoint
    • A sigmoid curve models the onset of the effect after the switchpoint, applied to the selected effect components

Feel free to share any feedback or suggestions! I'm happy to refine the model or explore extensions based on your input.


📚 Documentation preview 📚: https://causalpy--480.org.readthedocs.build/en/480/

@JeanVanDyk JeanVanDyk added enhancement New feature or request help wanted Extra attention is needed labels May 28, 2025
Copy link

codecov bot commented May 30, 2025

Codecov Report

Attention: Patch coverage is 8.10811% with 34 lines in your changes missing coverage. Please review.

Project coverage is 93.23%. Comparing base (6a9e3ad) to head (ee701f2).
Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
causalpy/pymc_models.py 8.10% 34 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #480      +/-   ##
==========================================
- Coverage   94.67%   93.23%   -1.44%     
==========================================
  Files          32       32              
  Lines        2196     2233      +37     
==========================================
+ Hits         2079     2082       +3     
- Misses        117      151      +34     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@JeanVanDyk JeanVanDyk force-pushed the intervention-time-estimator branch from b7b91de to ee701f2 Compare May 30, 2025 09:50
@JeanVanDyk
Copy link
Author

Note

In my last PR, I added functionality allowing users to specify priors for the effect of the intervention. This provides more flexibility for Bayesian modeling and allows users to incorporate domain knowledge directly into the inference process.

@drbenvincent drbenvincent marked this pull request as draft May 31, 2025 05:28
Copy link
Collaborator

@drbenvincent drbenvincent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @JeanVanDyk. I'm very excited about the new addition, thanks for putting it together. Bear with me if I'm sometimes slow to reply - busy dad life!

At this point, could I request that you put together an ipynb file that showcases the functionality. I'm assuming this PR will evolve and go through a couple of iterations, so this notebook will end up being a new docs page to help users understand the new functionality. If you edit the ./docs/source/notebooks/index.md file, you can add the notebook name under the interrupted time series section. That way the notebook will render if you either build the docs locally, but it should also render in the remote docs preview build.

Did you still want feedback on the GitHub Discussion at this point?

Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@JeanVanDyk
Copy link
Author

Simplified .fit() API

The InterventionTimeEstimator model has been streamlined. The .fit() method now only takes:

  • X: covariates
  • y: observed values
  • coords: coordinate labels for dimensions

Cleaner and more straightforward to use.


Optional Prior Specification

You can now pass a priors dictionary at init time to control intervention effects:

  • "level" and "trend"[mu, sigma] (Normal prior)
  • "impulse"[mu, sigma1, sigma2]
    • mu: amplitude mean (Normal)
    • sigma1: amplitude std (Normal)
    • sigma2: decay rate std (HalfNormal)

If an effect is omitted or an empty list is provided, the model uses default priors.


Support for Inferred Intervention Time

InterruptedTimeSeries now works with models that infer the treatment time.

  • Before fit():

    • If treatment_time is None or a Tuple, the model uses the full dataset to infer the switchpoint.
    • treatment_time is passed to the model via set_time_range().
  • After fit():

    • The inferred switchpoint is retrieved.
    • data_pre is reset accordingly for prediction, as in the standard flow.

Timeline Requirement

The time column t must be the last column of X (excluding y).
This is how the model locates the timeline internally.


Extras and Ongoing Work

  • Included a demo notebook for quick experimentation.
  • Currently working on:
    • More robust input_validation() inside InterruptedTimeSeries.
    • Supporting timelines expressed as datetime objects.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants