Document Celery subinterpreter task seams (#66)

[leynos]

Summary

This branch documents the coupling point between the Celery worker scaffold and the Python 3.14 interpreter-pool executor so future CPU-bound task authors can find the intended intra-task fan-out path.

Closes #66.

Review walkthrough

• Start with episodic/worker/tasks.py to see the task-author pattern for build_cpu_task_executor_from_environment() and map_ordered(...).
• Then review episodic/worker/runtime.py for the boundary between Celery pool configuration and task-level interpreter-pool knobs.
• Finish with docs/adr/adr-003-celery-worker-scaffold.md for the architectural cross-reference to the Python 3.14 concurrent-interpreter decision.

Validation

• make fmt: passed.
• make check-fmt: passed.
• make markdownlint: passed.
• make nixie: passed.
• make lint: passed.
• make typecheck: passed.
• make test: final rerun passed, reporting 477 passed and 3 skipped.

Notes

• Earlier full make test attempts hit transient pytest-timeout setup errors in py-pglite-backed async fixtures. The affected tests passed when rerun directly before the final full-suite pass.

Summary by Sourcery

Clarify how Celery worker runtime configuration relates to CPU-bound task interpreter-pool settings and document the intended task-level fan-out path via concurrent interpreters.

Documentation:

• Expand WorkerRuntimeConfig docstring to describe Celery-level dispatch concerns and explicitly distinguish task-level interpreter-pool environment variables and their executor adapter.
• Update the Celery worker scaffold ADR to document optional interpreter-pool fan-out for CPU-bound tasks and cross-reference the Python 3.14 concurrent-interpreters decision and related modules.

[coderabbitai]

Summary

This pull request resolves issue #66 by documenting the integration seam between Celery task workers and Python 3.14's subinterpreter-pool executor, enabling CPU-bound tasks to leverage intra-process fan-out patterns.

Key changes

Documentation

• Expanded ADR-003 (Celery worker scaffold) to document optional interpreter-pool delegation for CPU tasks, controlled via EPISODIC_USE_INTERPRETER_POOL, with cross-references to the concurrent-interpreters module and Python 3.14 upgrade plan.
• Added detailed module-level docstring to episodic/worker/tasks.py demonstrating the task-author pattern: calling build_cpu_task_executor_from_environment() and using map_ordered(...) within task bodies.
• Expanded WorkerRuntimeConfig docstring in episodic/worker/runtime.py to clarify the configuration boundary—interpreter-pool environment variables (EPISODIC_USE_INTERPRETER_POOL, EPISODIC_INTERPRETER_POOL_MIN_ITEMS) are task-level, not Celery worker-pool configuration.
• Updated docs/developers-guide.md with a new Celery worker runtime subsection showing CPU-executor construction, ordered async execution, and metrics export through CpuTaskExecutorMetricsPort.
• Added reference in the Python 3.14 execplan document noting that PR #99 closed the Celery integration gap.

Core implementation

• Refactored build_cpu_task_executor_from_environment() to accept an explicit environ mapping (defaulting to os.environ) and optional metrics sink, improving testability and observability.
• Enhanced InterpreterPoolCpuTaskExecutor with injected metrics and clock, an _is_shutdown guard, and rewritten shutdown() to atomically set state and hold locks for the full executor shutdown, preventing race conditions.
• Created episodic/metrics_ports.py defining shared BoundedMetricsPort and BoundedValueMetricsPort contracts, centralising metrics adapters for use by CPU executors and other components.
• Updated DefaultWeightingStrategy to forward metrics to the executor builder and refactored episodic/qa/chrono.py to reuse the shared metrics protocols.

Testing

• Extracted shared test fakes (FakeIoDiagnostic, FakeCpuDiagnostic, BlockingMapExecutor) into tests/conftest.py to reduce module length and support multiple test modules.
• Added two async integration tests in tests/test_worker_service_scaffold.py validating CPU task fan-out with environment-driven executor selection and fallback behaviour.
• Created tests/test_interpreter_executor_lifecycle.py with Hypothesis-based property tests covering shutdown races, sequential operations, concurrent maps, and failure scenarios.
• Created tests/test_interpreter_executor_observability.py validating lifecycle metrics recording, executor-selection signals, and metrics forwarding through the executor builder.
• Added tests/test_worker_interpreter_task_integration.py demonstrating eager Celery task execution with environment-driven interpreter-pool fan-out.
• Refactored tests/test_interpreter_executor.py to use property-based testing for order preservation and updated builder tests to pass explicit environ dictionaries, avoiding process-environment mutations.

Related artefacts

• ADR-003: Celery worker scaffold (updated with interpreter-pool consequences and cross-references)
• Execplan: Python 3.14 concurrent-interpreters upgrade (docs/execplans/upgrade-python-to-3-14-adopt-concurrent-interpreters.md)
• Issue #66: Reconcile Celery worker scaffold with Python 3.14 subinterpreter concurrency

Verification

All checks passed: make fmt, check-fmt, markdownlint, nixie, lint, typecheck. Test suite reported 477 passed, 3 skipped.

Walkthrough

Reconcile the Celery worker scaffold with Python 3.14 subinterpreter concurrency by establishing a metrics foundation, hardening executor lifecycle with locked state and injection points, documenting the task-level integration seam, and validating the pattern with comprehensive tests covering ordering, lifecycle races, observability and eager Celery invocation.

Changes

Celery–subinterpreter concurrency reconciliation

Layer / File(s)	Summary
Shared bounded-metrics abstractions episodic/metrics_ports.py	New module defines BoundedMetricsPort with counter and latency-observation methods, BoundedValueMetricsPort extending it with scalar observations, and frozen no-op dataclass implementations for both.
Chrono metrics integration episodic/qa/chrono.py	ChronoMetricsPort extends the shared BoundedMetricsPort; _NoopChronoMetrics inherits from NoopBoundedMetrics to eliminate duplicate stub code.
Executor API enhancement with metrics and environ injection episodic/concurrent_interpreters.py	Export CpuTaskExecutorMetricsPort, introduce monotonic clock protocol and production implementation, define module no-ops, and extend build_cpu_task_executor_from_environment to accept optional environ (defaulting to os.environ) and metrics parameters; builder records executor-selection reasons to metrics.
Executor lifecycle hardening and shutdown semantics episodic/concurrent_interpreters.py	Refactor InterpreterPoolCpuTaskExecutor to accept injected metrics and clock; replace non-reentrant lock with threading.RLock; add _is_shutdown guarded state with locked checks; implement locked lazy pool creation recording outcomes to metrics and emitting logs; rewrite atomic shutdown() to mark shutdown, clear executor, conditionally shut it down with latency recording; update map_ordered() to check shutdown (raising if shut down), return [] for empty inputs, run mapping in background thread whilst holding executor lock during map invocation, record map-call and item-count metrics only on success, and log/re-raise exceptions.
Weighting strategy metrics wiring episodic/canonical/adapters/weighting.py	DefaultWeightingStrategy.__init__ accepts new metrics parameter; executor builder call passes explicit os.environ and metrics=metrics when cpu_executor not supplied.
Worker config and task docstrings episodic/worker/runtime.py, episodic/worker/tasks.py	WorkerRuntimeConfig docstring adds Notes section clarifying task-level scope of interpreter-pool flags; tasks.py module docstring expands to describe Celery seam pattern and policy for EPISODIC_* environment variables; add docstrings to _parse_bool, _parse_positive_int, _parse_pool documenting expected types and RuntimeError on invalid input.
ADR decision and execplan retrospective docs/adr/adr-003-celery-worker-scaffold.md, docs/execplans/upgrade-python-to-3-14-adopt-concurrent-interpreters.md	ADR-003 Consequences adds bullet documenting optional inner fan-out delegation via InterpreterPoolCpuTaskExecutor controlled by EPISODIC_USE_INTERPRETER_POOL and tuned with EPISODIC_INTERPRETER_POOL_MIN_ITEMS / EPISODIC_INTERPRETER_POOL_MAX_WORKERS; execplan adds retrospective noting PR #99 closed the Celery integration gap with task-level guidance and tests.
Developer guide Celery CPU-task subsection docs/developers-guide.md	New subsection documents implementing CPU-bound episodic.cpu tasks: build executor from environment variables, execute ordered async work via map_ordered(...), optionally shutdown in finally block; clarify interpreter-pool flag semantics and CpuTaskExecutorMetricsPort wiring via DefaultWeightingStrategy(metrics=...) or direct builder parameter.
Surrounding documentation paragraph reflows docs/developers-guide.md, docs/episodic-podcast-generation-system-design.md	Minor rewraps across unrelated sections (lint guidance, migrations narrative, schema/validation descriptions, orchestration testing bullets, Chrono metrics TEI constraint, DefaultWeightingStrategy diagram, roadmap item description, effective_from_episode_id anchor behaviour).
User guide observability documentation docs/users-guide.md	Add observability note describing CPU-task executor metrics export via CpuTaskExecutorMetricsPort (extending BoundedValueMetricsPort); list bounded-label signals (executor-selection, interpreter-pool lifecycle, map item count, shutdown latency); instruct ingestion pipelines to wire same metrics sink to DefaultWeightingStrategy(metrics=...).
Project Hecate configuration pyproject.toml	Add episodic.metrics_ports to domain_ports prefix list.
Shared test fixtures, fakes and helpers tests/conftest.py	Add FakeIoDiagnostic and FakeCpuDiagnostic callable dataclass fakes recording diagnostic fields and returning canned results; add integer fan-out helpers double_worker_value and square_executor_value; introduce BlockingMapExecutor test double blocking map() until release event; add cpu_task_inner_fan_out async helper building executor from environment, running ordered double_worker_value mapping, invoking shutdown in finally; add runtime_environ fixture returning minimal eager Celery environment dict; add captured_interpreter_pool_workers fixture enabling interpreter-pool via environment and monkeypatching pool creation to capture requested max_workers.
Executor property and builder tests tests/test_interpreter_executor.py	Convert fixed inline/interpreter-backed ordering tests to Hypothesis property tests generating arbitrary integer lists and task choices; introduce _affine and _ORDERED_MAP_TASKS registry; import _square from conftest; patch pool creation to controlled ThreadPoolExecutor in finally block; refactor builder tests to pass explicit environ dict and _capability_check callable instead of monkeypatching process environment.
Executor shutdown and lifecycle tests tests/test_interpreter_executor.py, tests/test_interpreter_executor_lifecycle.py	Add three async lifecycle tests: shutdown() waits for active map_ordered() to complete, map_ordered() after shutdown raises RuntimeError without creating pools, shutdown() is idempotent; introduce four generated tests via Hypothesis: shutdown-race varying release timing, lifecycle-sequences of map/shutdown operations, concurrent-maps-then-shutdown verifying terminal RuntimeError, failure-sequences validating propagated ValueError and terminal shutdown state.
Executor observability and selection tests tests/test_interpreter_executor_observability.py	Add fake metrics, clock and logger capturing events in-memory; implement test_interpreter_executor_records_lifecycle_observability asserting map-call counters, item-count observations, creation-outcome and shutdown-latency metrics, log message substrings; implement test_builder_records_executor_selection_metrics covering three scenarios (feature flag disabled → inline, capability fails → unavailable, capability succeeds → interpreter-pool selected), asserting selection counters; implement test_builder_passes_metrics_to_interpreter_executor verifying metrics forwarding into interpreter executor.
Weighting strategy metrics forwarding test tests/test_ingestion_weighting.py	Add _RecordingCpuTaskExecutorMetrics dataclass test double capturing counter increments; add test that monkeypatches executor-builder factory to capture metrics parameter and asserts DefaultWeightingStrategy(metrics=metrics) forwards the same instance to the builder.
Service scaffold test refactoring tests/test_worker_service_scaffold.py	Remove local diagnostic fakes (_FakeIoDiagnostic, _FakeCpuDiagnostic) and runtime-environ builder; import shared fakes and cpu_task_inner_fan_out from conftest; refactor existing tests accepting injected runtime_environ fixture; add async test validating cpu_task_inner_fan_out ordering and interpreter-pool worker creation via captured_interpreter_pool_workers; add async test deleting EPISODIC_USE_INTERPRETER_POOL and asserting fallback to inline execution when pool creation fails.
Eager Celery integration test with environment executor tests/test_worker_interpreter_task_integration.py	New test module with test_eager_cpu_task_body_uses_environment_executor_pattern: creates Celery app from runtime config, registers eager task validating payload as JSON integer list, constructs CPU executor from os.environ, executes ordered abs mapping via asyncio.run, invokes shutdown in finally, returns mapped results; test invokes task eagerly with [-1, 3, -5] and asserts result [1, 3, 5] with captured_interpreter_pool_workers exactly [2].

---

Subinterpreter seams sewn into Celery's prefork loom, 🧵
Shutdown states atomic, metrics in every room,
Doctstrings guide the path from task to pool,
Tests race through lifecycles—concurrency stays cool. ✓

---

Possibly related PRs

• leynos/episodic#39: Earlier refactoring of InterpreterPoolCpuTaskExecutor and executor builder—this PR extends that foundation with lifecycle hardening, metrics injection and comprehensive tests.

📋 Issue Planner

Built with CodeRabbit's Coding Plans for faster development and fewer bugs.

View plan used: #66

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch issue-66-reconcile-celery-worker-scaffold-with-subinterpreter

[sourcery-ai]

Reviewer's Guide

Documents and clarifies the boundary between Celery worker runtime configuration and the Python 3.14 interpreter-pool executor, and cross-links ADR/docs so CPU-bound task authors can discover and use the interpreter-pool fan-out path correctly.

File-Level Changes

Change	Details	Files
Clarify Celery worker runtime configuration vs. interpreter-pool task-level configuration.	Expanded the WorkerRuntimeConfig docstring to describe that it only governs Celery-level dispatch concerns such as broker, backend, pool selection, and concurrency. Documented the EPISODIC_USE_INTERPRETER_POOL, EPISODIC_INTERPRETER_POOL_MIN_ITEMS, and EPISODIC_INTERPRETER_POOL_MAX_WORKERS environment variables as task-level knobs for CPU-bound workloads. Explained that CPU-bound task code should use build_cpu_task_executor_from_environment() to access interpreter-pool execution, and why these settings are intentionally excluded from WorkerRuntimeConfig.	episodic/worker/runtime.py
Document interpreter-pool usage and cross-reference concurrent-interpreter architecture in the Celery worker ADR.	Extended the ADR to describe that CPU tasks in prefork workers may delegate inner fan-out to InterpreterPoolCpuTaskExecutor, controlled by interpreter-pool-related environment variables. Added references from the ADR to the Python 3.14 concurrent-interpreters execution plan and to the concurrent_interpreters module. Reinforced the architectural seam between Celery worker topology/runtime and interpreter-pool-based intra-task parallelism for selected workloads.	docs/adr/adr-003-celery-worker-scaffold.md
Surface the task-author pattern and seam for CPU-bound tasks using the interpreter pool.	Documented (or adjusted) the expected usage of build_cpu_task_executor_from_environment() and map_ordered(...) as the standard pattern for CPU-bound Celery tasks that fan out work via the interpreter pool. Aligned task-level documentation with the runtime/ADR terminology so future task authors can discover the intended intra-task fan-out path.	episodic/worker/tasks.py

Assessment against linked issues

Issue	Objective	Addressed	Explanation
#66	Provide a clear integration seam and example in Celery CPU-bound tasks (e.g., in tasks.py) for using build_cpu_task_executor_from_environment()/InterpreterPoolCpuTaskExecutor for intra-process subinterpreter fan-out.	✅
#66	Update ADR-003 to document that CPU tasks in prefork workers may optionally fan out via InterpreterPoolCpuTaskExecutor and to cross-reference the Python 3.14 concurrent-interpreters decision and implementation.	✅
#66	Document in the worker runtime/config (WorkerRuntimeConfig) that EPISODIC_USE_INTERPRETER_POOL and related variables are task-level knobs for intra-task parallelism, not Celery pool-level settings, and therefore are not surfaced via WorkerRuntimeConfig.	✅

Possibly linked issues

• Reconcile Celery worker scaffold with Python 3.14 subinterpreter concurrency #66: PR adds docs and runtime/task comments that reconcile Celery worker scaffold with interpreter-pool concurrency per issue.

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Sorry @leynos, you have reached your weekly rate limit of 2500000 diff characters.

Please try again later or upgrade to continue using Sourcery

[leynos]

@coderabbitai Please investigate the cause of the following issue using codegraph exploration and research, identify a fix and provide an AI coding agent prompt for the fix:

Run make lint
PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 UV_CACHE_DIR=.uv-cache UV_TOOL_DIR=.uv-tools /opt/hostedtoolcache/uv/0.11.15/x86_64/uv sync --group dev
Resolved 105 packages in 1ms
Checked 102 packages in 1ms
PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 UV_CACHE_DIR=.uv-cache UV_TOOL_DIR=.uv-tools /opt/hostedtoolcache/uv/0.11.15/x86_64/uv run python -m episodic.architecture
PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 UV_CACHE_DIR=.uv-cache UV_TOOL_DIR=.uv-tools /opt/hostedtoolcache/uv/0.11.15/x86_64/uv run ruff check
All checks passed!
PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 UV_CACHE_DIR=.uv-cache UV_TOOL_DIR=.uv-tools /opt/hostedtoolcache/uv/0.11.15/x86_64/uv tool run --python pypy --from 'git+https://github.com/leynos/pylint-pypy-shim.git@726d09f968b4d729ee4b29c71fc732e744854f3b' pylint-pypy alembic episodic openai_test_types.py tests
Downloading pypy-3.11.15-linux-x86_64-gnu (download) (33.3MiB)
 Downloaded pypy-3.11.15-linux-x86_64-gnu (download)
   Updating https://github.com/leynos/pylint-pypy-shim.git (726d09f968b4d729ee4b29c71fc732e744854f3b)
    Updated https://github.com/leynos/pylint-pypy-shim.git (726d09f968b4d729ee4b29c71fc732e744854f3b)
   Building pylint-pypy-shim @ git+https://github.com/leynos/pylint-pypy-shim.git@726d09f968b4d729ee4b29c71fc732e744854f3b
      Built pylint-pypy-shim @ git+https://github.com/leynos/pylint-pypy-shim.git@726d09f968b4d729ee4b29c71fc732e744854f3b
Installed 8 packages in 6ms
************* Module test_worker_service_scaffold
tests/test_worker_service_scaffold.py:7:0: C0302: Too many lines in module (469/400) (too-many-lines)

------------------------------------
Your code has been rated at 10.00/10

make: *** [Makefile:75: lint] Error 16
Error: Process completed with exit code 2.

[leynos]

@coderabbitai what?! You're not supposed to be telling the developer how to bypass the lint

[leynos]

@coderabbitai review

[coderabbitai]

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[leynos]

@coderabbitai review

[coderabbitai]

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[leynos]

@coderabbitai Have the following failed checks now been resolved?

If further work is required, please provide an AI agent prompt for the remaining work to be done to address these failures.

Do not treat warnings as optional or aspirational. Where a change is out of scope for this PR, propose a GitHub issue unless one exists already.

❌ Failed checks (1 error, 2 warnings)

Check name	Status	Explanation	Resolution
Unit Architecture	❌ Error	PR promotes build_cpu_task_executor_from_environment() which directly calls os.getenv() rather than accepting environ as injectable parameter, hiding environment-access fallibility.	Refactor function to accept environ as parameter; inject capability detection as callable; update pattern and tests to use proper dependency injection at boundaries.
Docstring Coverage	⚠️ Warning	Docstring coverage is 78.57% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.
Concurrency And State	⚠️ Warning	Race condition in InterpreterPoolCpuTaskExecutor: lock released before executor.shutdown(wait=True) allows concurrent map_ordered() to race. Tests lack concurrent shutdown scenarios.	Synchronise map_ordered() and shutdown() atomically. Add tests for concurrent shutdown + map_ordered and out-of-order invocation scenarios.

[leynos]

@coderabbitai Please investigate the cause of the following issue using codegraph exploration and research, identify a fix and provide an AI coding agent prompt for the fix:

[8](https://github.com/leynos/episodic/actions/runs/26409273751/job/77739884701?pr=99#step:14:1269)
tests/test_workflow_provision_doks.py: 1 warning
  /home/runner/work/episodic/episodic/.venv/lib/python3.14/site-packages/slipcover/__main__.py:33: DeprecationWarning: This process (pid=17264) is multi-threaded, use of fork() may lead to deadlocks in the child.
    if (pid := original_fork(*pargs, **kwargs)):

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
===================== 🚀 py-pglite Framework Isolation Tips =====================
For perfect framework isolation, try these patterns:
  pytest -m sqlalchemy -p no:django     # Pure SQLAlchemy tests
  pytest -m django                      # Pure Django tests
  pytest testing-patterns/sqlalchemy/   # Directory isolation
  pytest testing-patterns/django/       # Directory isolation

See pytest.ini for more elegant patterns! ✨
=========================== short test summary info ============================
FAILED tests/steps/test_http_service_scaffold_steps.py::test_granian_serves_health_endpoints - self = <sqlalchemy.engine.base.Connection object at 0x7fdfb18d0050>
engine = Engine(***/postgres?host=%2Ftmp%2Fpy-pglite-19949-1b5948e8)
connection = None, _has_events = None, _allow_revalidate = True
_allow_autobegin = True

    def __init__(
        self,
        engine: Engine,
        connection: Optional[PoolProxiedConnection] = None,
        _has_events: Optional[bool] = None,
        _allow_revalidate: bool = True,
        _allow_autobegin: bool = True,
    ):
        """Construct a new Connection."""
        self.engine = engine
        self.dialect = dialect = engine.dialect
    
        if connection is None:

engine = <sqlalchemy.ext.asyncio.engine.AsyncEngine object at 0x7fdfb184b750>

    async def _wait_for_engine_ready(engine: AsyncEngine) -> None:
        """Wait for the helper-managed py-pglite engine to accept connections."""
        max_attempts = 30
        delay_seconds = 0.1
        for attempt in range(1, max_attempts + 1):
            try:
                async with engine.connect() as connection:
                    await connection.execute(sa.text("SELECT 1"))
            except sa_exc.OperationalError as exc:
                if attempt == max_attempts:
                    msg = (
                        f"py-pglite engine not ready after {max_attempts} "
                        f"attempts ({delay_seconds}s apart)"
                    )
>                   raise RuntimeError(msg) from exc
E                   RuntimeError: py-pglite engine not ready after 30 attempts (0.1s apart)

tests/fixtures/database.py:74: RuntimeError
====== 1 failed, 684 passed, 2 skipped, 688 warnings in 345.86s (0:05:45) ======
Error: Process completed with exit code 1.

[leynos]

@coderabbitai review

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@episodic/metrics_ports.py`:
- Line 9: Remove the forbidden future import by deleting the line "from
__future__ import annotations" wherever it appears (e.g., in the
episodic.metrics_ports and episodic.concurrent_interpreters modules) so the code
no longer uses the banned future import for Python >= 3.14; ensure no other code
relies on postponed evaluation (update type annotations to use standard runtime
annotations or string literals if necessary) and run the test/linter suite to
confirm no remaining references.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 224f9792-d912-4e95-b41c-6e5b2ec5b974

📥 Commits

Reviewing files that changed from the base of the PR and between db181d5 and 0c7520a.

📒 Files selected for processing (12)

• docs/developers-guide.md
• docs/episodic-podcast-generation-system-design.md
• docs/users-guide.md
• episodic/canonical/adapters/weighting.py
• episodic/concurrent_interpreters.py
• episodic/metrics_ports.py
• episodic/qa/chrono.py
• pyproject.toml
• tests/test_ingestion_weighting.py
• tests/test_interpreter_executor_lifecycle.py
• tests/test_interpreter_executor_observability.py
• tests/test_worker_interpreter_task_integration.py

💤 Files with no reviewable changes (3)

• tests/test_worker_interpreter_task_integration.py
• tests/test_interpreter_executor_observability.py
• tests/test_interpreter_executor_lifecycle.py

[coderabbitai]

Actionable comments posted: 5

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

docs/episodic-podcast-generation-system-design.md (1)

877-889: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Wrap this paragraph at 80 columns.

Lines 877, 878, 879, 881, 882, 884, 885, and 888 exceed the 80-column limit. Reflow the entire paragraph to comply.

Triage: [type:docstyle]

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/episodic-podcast-generation-system-design.md` around lines 877 - 889,
Reflow the paragraph describing the 2.4.1 roadmap item to wrap at 80 columns and
fix all over-long lines; edit the text that mentions
StructuredGenerationPlanner, LLMPort, ExecutionPlan,
GenerationOrchestrationConfig, StructuredPlanningOrchestrator, ToolExecutorPort,
ShowNotesToolExecutor, GuestBiosToolExecutor, RoutingToolExecutor, and LangGraph
so each sentence is wrapped to <=80 characters without changing technical names
or meaning and keep the same sequence of ideas (planner calls LLMPort for strict
JSON -> parse to ExecutionPlan and record planning vs execution model ->
orchestrator executes via ToolExecutorPort with the two concrete executors ->
RoutingToolExecutor dispatches by ActionKind -> LangGraph wrapper remains
in-process and limited to plan -> execute -> finish until later features land).

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/episodic-podcast-generation-system-design.md`:
- Around line 552-555: Reflow the caption beginning "Caption: Speech synthesis
entity relationships." so no line exceeds 80 columns: break the sentence into
multiple lines (wrapping between words, not mid-word) so the full caption
"Pronunciation entries have one or more realizations; voice personas and
provider capabilities are selected for speech render requests; each speech
render request produces one or more speech render artefacts." is wrapped into
lines <=80 chars while preserving punctuation and meaning.
- Around line 1544-1546: The sentence starting with "resolve these bindings and
will snapshot selected revisions into ingestion-bound `source_documents`,
preserving reproducible TEI provenance while allowing independent document reuse
across jobs." exceeds 80 columns; reflow this sentence into multiple lines
wrapped at ~80 characters, preserving the inline code marker `source_documents`
and the original punctuation and meaning across the new lines.
- Around line 420-426: Paragraph exceeds the 80-column limit; reflow the entire
paragraph to wrap at 80 characters per line while preserving wording,
punctuation, and inline code/backticks for `TTSPort` and `DialogueSpeechPort`.
Keep the same sentences and meaning about the speech synthesis boundary,
single-speaker `TTSPort` for ordinary narration and partial regeneration, and
the optional `DialogueSpeechPort` for providers (e.g., Inworld Realtime,
ElevenLabs) that render multi‑speaker dialogue; ensure each new line is ≤80
columns and maintain readability and hyphenation where appropriate.
- Around line 720-721: The second sequence line exceeds 120 columns; shorten or
wrap the label for the message from DefaultWeightingStrategy to itself so the
code block stays ≤120 chars—e.g., replace
"build_cpu_task_executor_from_environment(os.environ, metrics=metrics)" with a
shortened label like "build_cpu_task_executor_from_environment(...,
metrics=metrics)" or split the message into two lines (keep the
Caller->>DefaultWeightingStrategy line unchanged and adjust the
DefaultWeightingStrategy->>DefaultWeightingStrategy message).
- Around line 494-499: The paragraph starting with "Alert on synthesis failures
and capability mismatches using aggregate metrics" exceeds the 80-column limit
and uses American "percent"; reflow this sentence into multiple lines under 80
characters and change "5 percent" to British "5 per cent", preserving the rest
of the wording (including the clause "page when the failure rate exceeds 5 per
cent over 15 minutes or five consecutive renders fail, warn when repeated
unsupported-capability diagnostics reach three in 30 minutes, and page when an
approved"). Ensure line breaks occur at natural phrase boundaries so readability
is maintained.

---

Outside diff comments:
In `@docs/episodic-podcast-generation-system-design.md`:
- Around line 877-889: Reflow the paragraph describing the 2.4.1 roadmap item to
wrap at 80 columns and fix all over-long lines; edit the text that mentions
StructuredGenerationPlanner, LLMPort, ExecutionPlan,
GenerationOrchestrationConfig, StructuredPlanningOrchestrator, ToolExecutorPort,
ShowNotesToolExecutor, GuestBiosToolExecutor, RoutingToolExecutor, and LangGraph
so each sentence is wrapped to <=80 characters without changing technical names
or meaning and keep the same sequence of ideas (planner calls LLMPort for strict
JSON -> parse to ExecutionPlan and record planning vs execution model ->
orchestrator executes via ToolExecutorPort with the two concrete executors ->
RoutingToolExecutor dispatches by ActionKind -> LangGraph wrapper remains
in-process and limited to plan -> execute -> finish until later features land).

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 0a3c4edf-190b-45ca-92e6-5d63d53b09e1

📥 Commits

Reviewing files that changed from the base of the PR and between 0c7520a and b710a05.

📒 Files selected for processing (19)

• docs/adr/adr-003-celery-worker-scaffold.md
• docs/developers-guide.md
• docs/episodic-podcast-generation-system-design.md
• docs/execplans/upgrade-python-to-3-14-adopt-concurrent-interpreters.md
• docs/users-guide.md
• episodic/canonical/adapters/weighting.py
• episodic/concurrent_interpreters.py
• episodic/metrics_ports.py
• episodic/qa/chrono.py
• episodic/worker/runtime.py
• episodic/worker/tasks.py
• pyproject.toml
• tests/conftest.py
• tests/test_ingestion_weighting.py
• tests/test_interpreter_executor.py
• tests/test_interpreter_executor_lifecycle.py
• tests/test_interpreter_executor_observability.py
• tests/test_worker_interpreter_task_integration.py
• tests/test_worker_service_scaffold.py