Skip to content

Stabilize hybrid jump-diffusion variance test - #1519

Merged
ChrisRackauckas merged 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:agent/stabilize-hybrid-variance-test
Jul 26, 2026
Merged

Stabilize hybrid jump-diffusion variance test#1519
ChrisRackauckas merged 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:agent/stabilize-hybrid-variance-test

Conversation

@ChrisRackauckas-Claude

@ChrisRackauckas-Claude ChrisRackauckas-Claude commented Jul 21, 2026

Copy link
Copy Markdown
Member

Important

Ignore this PR until it has been reviewed by @ChrisRackauckas.

Summary

  • Increase the Monte Carlo sample count for the Brownian-plus-Poisson variance check from 500 to 1500.
  • Preserve both the expected variance formula and the existing 15% relative tolerance.

Why

The StableRNG supplied to HybridProblem controls the jump process, while SRIW1 draws Brownian increments from the task RNG. The sample-variance estimator therefore changes with the outer test RNG. At 500 trials its approximate relative standard error is sqrt(2 / (500 - 1)) = 6.3% (and can be slightly larger for the jump-diffusion distribution), so the 15% acceptance band is only about 2.4 standard errors wide.

The outer RNG recorded by SciML/ModelingToolkit.jl#4793 produced a variance of 33.82356404805082, just outside the lower bound of 34.0. Full-file replay reproduced that exact value on both the clean ModelingToolkit base (f6c5c80d) and PR head (bb9b3fdc). Their resolved dependency graphs were identical after normalizing local paths, including DiffRules 1.16.0, so the failure is independent of that PR.

At 1500 trials the approximate relative standard error falls to 3.7%, putting the unchanged 15% band more than four standard errors away. git blame traces the statistical test to ac967b29, introduced through #1386.

Local validation

  • Julia 1.12.6, exact recorded failing RNG and dependency graph, full hybrid_models.jl: 4218/4218 passed in 15m21.1s.
  • Julia 1.12.6, official GROUP=Hybrid harness on final rebased head 53e6bacb against base 730c79ea: 4218/4218 passed in 21m35.8s; process exit status 0.
  • Six 1500-trial task-RNG states passed; observed variance range was 38.32–44.64 for the expected value 40.0 and unchanged accepted range 34.0–46.0.
  • Runic 1.7.0 left the modified line range unchanged; git diff --check passed. The whole-file Runic check reports pre-existing drift in untouched portions of hybrid_models.jl, which this focused PR does not rewrite.

@ChrisRackauckas-Claude

ChrisRackauckas-Claude commented Jul 21, 2026

Copy link
Copy Markdown
Member Author

Final post-hold CI and clean-base classification:

Current revision and target result

PR #1519 is a single commit (53e6bacb) against current base 730c79ea, changing only test/simulation_and_solving/hybrid_models.jl. The required co-author trailer is present.

All 22 hosted checks are terminal: 18 passed, 4 failed, 0 pending. All three target Hybrid lanes passed:

The official local Hybrid group also passed on this exact head/base pair:

  • ReactionSystem Hybrid Solvers | 4218 pass / 4218 total
  • test time 21m35.8s; process exit status 0
  • raw log: /home/crackauc/sandbox/tmp_20260715_055953_80193/audit/catalyst-pr1519-rebased2-53e6bac-group-hybrid-julia-1.12.6.log
  • SHA-256: 1c52957b4d70005ece4102ad6c74d7c014c380cf73c5ed2396d2f8798dd32501

Four unrelated failures

None is caused by this PR's two-line Hybrid-test hunk:

  1. Modeling / Julia lts: clean f3489f88 reproduces DSL Options | 308 pass, 9 fail, 1 broken. Existing tracking/fix path: Catalyst #1502, Catalyst #1503, ModelingToolkit #4211, and ModelingToolkit #4714.
  2. Modeling / Julia 1 and pre: clean f3489f88 reproduces Conservation Laws | 242 pass, 7 error, 17 broken; the seven errors are stale @test_broken assertions that unexpectedly pass. The focused correction is Catalyst #1512.
  3. QA / Julia 1: the self-hosted runner lost communication; its log blob returns BlobNotFound. Both completed QA code lanes passed: Julia lts and Julia pre.

The earlier public-reexport QA defect was clean-base-confirmed at the a90b5425 (#1501) boundary and fixed by #1520, merged as f7164c27 and included in this PR's current base. Its exact 309-name allowlist keeps check_reexports enabled, has no missing/extra names, and passed local QA on Julia 1.10.11, 1.12.6, and 1.13.0-rc1. Its completed hosted QA lts/pre lanes also passed; post-hold #1520 classification.

Detailed clean-base commands, package graph, results, and verified raw-log hashes are indexed at /home/crackauc/sandbox/tmp_20260715_055953_80193/audit/logs/catalyst-master-f348-ci/README.md; all 25 entries in the adjacent SHA256SUMS verify successfully.

The exact recorded #4793 RNG full-file replay passed 4218/4218: /home/crackauc/sandbox/tmp_20260715_055953_80193/audit/catalyst-pr1519-exact-ci-rng-full-file-julia-1.12.6.log (sha256:a8ca0faffbc5f72836f25613284079f6fb4be2e3be6f143f32baad3fe314efd3).

@ChrisRackauckas-Claude
ChrisRackauckas-Claude force-pushed the agent/stabilize-hybrid-variance-test branch from 0362c03 to ca0285b Compare July 21, 2026 18:19
Increase the Monte Carlo sample count while preserving the variance tolerance.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
@TorkelE

TorkelE commented Jul 22, 2026

Copy link
Copy Markdown
Member

Still travelling, and will untik end of next week, but when I am back I will try to have a closer look at things again

@ChrisRackauckas
ChrisRackauckas marked this pull request as ready for review July 26, 2026 21:01
@ChrisRackauckas
ChrisRackauckas merged commit 53e6482 into SciML:master Jul 26, 2026
18 of 22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants