Skip to content

refactor(lineage-gc): split scheduler to tombstone-only (Phase 2) - #259

Merged
guangyu-reflexio merged 5 commits into
mainfrom
refactor/phase-2-premium-reclaimer-split
Jun 30, 2026
Merged

refactor(lineage-gc): split scheduler to tombstone-only (Phase 2)#259
guangyu-reflexio merged 5 commits into
mainfrom
refactor/phase-2-premium-reclaimer-split

Conversation

@guangyu-reflexio

@guangyu-reflexio guangyu-reflexio commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Phase 2 — Premium reclaimer split (OSS side)

Reduces the shared lineage GC daemon to tombstone-GC only (core local data hygiene), splitting the premium governance-retention reclaimer out to an enterprise Capability (see the enterprise PR).

What changed

  • LineageGCScheduler / maybe_start_lineage_gc reduced to tombstone-only — dropped the gc_governance_retention branch, the _is_governance_retention_enabled helper, and the GovernanceRetentionConfig import; gate is now if not cfg.lineage_gc.enabled. Tombstone GC is unchanged (same entity loop, cutoff, high-volume tripwire, per-org resilience) and still default-on in all modes — no tombstone/PII-retention regression.
  • Dead-knob warning — an OSS-only deployment that sets audit_events_retention_enabled=True now gets a one-time startup warning (the reclaimer lives in enterprise); the config field is documented enterprise-only.
  • gc_governance_retention stays an OSS abstract storage method (consumed by the enterprise capability via the public ctx.storage interface) + a new backend-agnostic cross-backend contract test.

Tests

  • Effect-level shed test: the reduced scheduler never calls gc_governance_retention even with retention enabled, while tombstone GC still runs.
  • Legacy-config path + LineageGCConfig().enabled is True default-flip tripwire.
  • Backend-agnostic gc_governance_retention contract test (SQLite via the shared fixture; auto-covers Supabase when that param is added).

Behavior-preserving for tombstone GC; design doc reviewed across 3 rounds (incl. a focused verification that caught + closed a managed-tenant silent-skip risk).

Summary by CodeRabbit

  • Bug Fixes
    • Lineage garbage collection now focuses on removing only expired tombstones, with clearer behavior when retention options are unset or disabled.
    • Enterprise-only retention settings are treated as inactive in unsupported deployments, and a startup warning is shown to prevent confusion about “no-op” retention.
  • Tests
    • Expanded unit and integration coverage for tombstone cleanup, governance-retention delete behavior (including batch limits and idempotency), and “dead-knob” warning scenarios.

… to enterprise scheduler)

The governance-retention gating logic was stripped from LineageGCScheduler;
its remaining valid coverage (tombstone gating, dead-knob warning, legacy
config, never-runs-governance-retention) already lives in test_gc_scheduler.py.
The 4 governance-specific tests here now fail against the reduced scheduler.
@coderabbitai

coderabbitai Bot commented Jun 30, 2026

Copy link
Copy Markdown

Review Change Stack

Important

Review skipped

No new commits to review since the last review.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 70271084-1401-403c-9546-3ca6772980f9

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

LineageGCScheduler now handles tombstone GC only. Governance-retention behavior is removed from scheduler ticking and startup gating, GovernanceRetentionConfig is documented as enterprise-only, and new tests cover the OSS warning path plus storage-level retention GC behavior.

Changes

Tombstone-only scheduler and governance-retention contract tests

Layer / File(s) Summary
Config docstring and scheduler tombstone-only refactor
reflexio/models/config_schema.py, reflexio/server/services/lineage/gc_scheduler.py
GovernanceRetentionConfig docstring now marks audit-event retention as enterprise-only. The lineage GC scheduler docs, imports, _gc_tick, and maybe_start_lineage_gc are updated to remove governance-retention GC and add the OSS dead-knob warning for audit_events_retention_enabled.
Scheduler unit tests: tombstone-only tick and dead-knob warnings
tests/server/services/lineage/test_gc_scheduler.py
Adds coverage for tombstone-only ticking, legacy configs without governance_retention, LineageGCConfig.enabled defaulting to True, and maybe_start_lineage_gc warning behavior across OSS and enterprise-style configurators.
Storage-contract integration tests for gc_governance_retention
tests/server/services/storage/test_storage_contract_gc_governance_retention.py
Adds integration tests for aged-event deletion, recent-event preservation, disabled no-op behavior, batch-limit enforcement, and idempotency after all eligible events are removed.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • ReflexioAI/reflexio#188: Introduces the lineage tombstone GC scheduler that this PR narrows to tombstone-only behavior.
  • ReflexioAI/reflexio#201: Changes LineageGCConfig default enablement, which is asserted by the new scheduler regression test.
  • ReflexioAI/reflexio#250: Adds governance-retention scheduler plumbing that this PR removes from the lineage GC path.

Poem

🐇 Hop, hop — the tombstones sweep by,
Old retention knobs now just sigh.
In OSS lands, a warning glows,
For enterprise magic that doesn’t grow.
Clean little burrows, neat and bright,
The scheduler hops through day and night.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 75.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: splitting lineage GC into tombstone-only scheduler behavior in Phase 2.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch refactor/phase-2-premium-reclaimer-split

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/server/services/lineage/test_gc_scheduler.py`:
- Around line 337-447: The dead-knob tests are triggering the real lineage GC
scheduler, which spawns background threads and leaks noise across tests. Update
the `_config`/`_ctx` test helpers and the `maybe_start_lineage_gc` call sites so
these cases only exercise the warning path without starting `_run_loop`; include
the missing `poll_interval_seconds` on the fake `lineage_gc` config or mock the
startup path so no daemon thread is created. Use the existing
`maybe_start_lineage_gc`, `_config`, and `_ctx` helpers to keep the tests
isolated.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: e3b51814-c7ec-4890-9e59-401b0d8b73ab

📥 Commits

Reviewing files that changed from the base of the PR and between 7f8f49b and e17d3c1.

📒 Files selected for processing (5)
  • reflexio/models/config_schema.py
  • reflexio/server/services/lineage/gc_scheduler.py
  • tests/server/services/lineage/test_gc_scheduler.py
  • tests/server/services/lineage/test_governance_retention_gates.py
  • tests/server/services/storage/test_storage_contract_gc_governance_retention.py
💤 Files with no reviewable changes (1)
  • tests/server/services/lineage/test_governance_retention_gates.py

Comment thread tests/server/services/lineage/test_gc_scheduler.py Outdated
…threads

Use lineage_gc_enabled=False in the three dead-knob warning tests that had
lineage_gc_enabled=True, so maybe_start_lineage_gc returns None immediately
after emitting the warning (the warning fires before the enabled gate) without
spawning a real LineageGCScheduler daemon thread.

Also fix pre-existing ruff N814 (SimpleNamespace imported as _NS alias) and
nine pre-existing pyright arg-type/assignment errors throughout the file.
@guangyu-reflexio

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 30, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant