feat(session): add retry endpoint for failed archive Phase 2 extraction#3340
Open
njuboy11 wants to merge 1 commit into
Open
feat(session): add retry endpoint for failed archive Phase 2 extraction#3340njuboy11 wants to merge 1 commit into
njuboy11 wants to merge 1 commit into
Conversation
When a session archive's Phase 2 (memory extraction) step fails —
typically due to transient VLM/LLM API errors, timeouts, or upstream
rate-limits — OpenViking currently writes a ``.failed.json`` marker
and gives up. Operators have no programmatic way to re-trigger the
extraction; the only options are to restart the service (which loses
all in-flight extraction state) or to manually delete the marker and
hope the next commit re-runs.
This change adds a first-class retry endpoint:
POST /api/v1/sessions/{session_id}/archives/{archive_id}/retry
The handler re-reads the archive's messages.jsonl, deletes the
``.failed.json`` marker if present, and re-runs ``_run_memory_extraction``
as a background task. The response carries a ``task_id`` so callers can
poll progress through the existing ``GET /tasks/{task_id}`` endpoint.
Three files touched:
* ``openviking/session/session.py``
- New ``skip_previous_archive_check`` keyword argument on
``_run_memory_extraction`` (default ``False`` — backwards
compatible). When ``True``, the call to
``_wait_for_previous_archive_done`` is skipped, which is necessary
for retries since the previous archive may itself lack ``.done``.
- New ``Session.retry_archive_extraction(archive_index)`` method
that performs the retry.
* ``openviking/server/routers/sessions.py``
- New ``POST /{session_id}/archives/{archive_id}/retry`` endpoint
that delegates to the service-layer wrapper.
* ``openviking/service/session_service.py``
- New ``SessionService.retry_archive`` wrapper that resolves the
session and forwards to ``retry_archive_extraction``.
Production validation
---------------------
Live-tested on OpenViking 0.4.10 with 19 failed archives:
- 19/19 POST requests returned HTTP 200
- 16 archives transitioned to ``.done`` with new ``memory_diff.json``
containing 47 ``add`` ops and 13 ``update`` ops across 1,280
committed messages
- 3 archives remained stuck due to persistent failures (LLM
``empty_response``, plugin API incompatibility, missing Phase 2
invocation entirely) — these are not addressed by this change but
are now easier to triage with the retry endpoint
- 1 archive self-healed mid-poll when its ``memory_diff.json`` and
``.done`` marker were both written 24 ms apart by the retry task,
confirming the polling path is consistent
Related issues
--------------
* volcengine#2936 — auto-commit threshold wiring
* volcengine#2476 — force flag to skip blocking failed archives
* volcengine#3297 — event-memory extraction fails when a
new page_id is encountered
Compatibility
-------------
* The ``skip_previous_archive_check`` kwarg defaults to ``False``;
existing callers see no behavioural change.
* The new endpoint is purely additive.
* No protocol-level changes; existing clients continue to work.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
When a session archive's Phase 2 (memory extraction) step fails — typically due to transient VLM/LLM API errors, timeouts, or upstream rate-limits — OpenViking currently writes a
.failed.jsonmarker and gives up. Operators have no programmatic way to re-trigger extraction; the only options are to restart the service (which loses all in-flight extraction state) or to manually delete the marker and hope the next commit re-runs.This change adds a first-class retry endpoint:
The handler re-reads the archive's
messages.jsonl, deletes the.failed.jsonmarker if present, and re-runs_run_memory_extractionas a background task. The response carries atask_idso callers can poll progress through the existingGET /tasks/{task_id}endpoint.Human Involvement
Type of Change
Changes Made
openviking/session/session.pyskip_previous_archive_check: bool = Falseon_run_memory_extraction. WhenTrue, the call to_wait_for_previous_archive_doneis skipped — necessary for retries since the previous archive may itself lack.done.Session.retry_archive_extraction(archive_index)method (≈70 lines) that performs the retry: re-reads messages, deletes.failed.json, creates a tracker task, schedules_run_memory_extractionwithskip_previous_archive_check=True.openviking/server/routers/sessions.pyPOST /{session_id}/archives/{archive_id}/retryendpoint that delegates to the service-layer wrapper. Uses the samerun_operationtelemetry path asextract_session.openviking/service/session_service.pySessionService.retry_archive(session_id, archive_id, ctx)wrapper that resolves the session and forwards toretry_archive_extraction.Diff stat
Testing
Production validation (OpenViking 0.4.10, 2026-07-18)
A batch of 19 failed archives was retried via the new endpoint:
.donewith newmemory_diff.jsonadd+ 13updateops across 1,280 committed messagesempty_response, plugin API incompat, orphan Phase 2)memory_diff.jsonand.doneboth written 24 ms apart by retry taskAll three modified files compile cleanly (
python3 -m py_compile) and the service starts normally (/healthreturns{"status":"ok"}).(Unit-test coverage for the new endpoint is not included in this PR; the production validation above was performed against a live 0.4.10 deployment. Happy to add pytest coverage as a follow-up if reviewers want.)
Checklist
Compatibility
skip_previous_archive_checkkwarg defaults toFalse; existing callers see no behavioural change.Additional Notes
Why endpoint and not a CLI subcommand?
Two options were considered:
ov retry-archive <sid> <archive_id>CLI subcommand — would only work when the operator has shell access to the server.extract_sessionendpoint, and is trivially invokable from the OpenClaw plugin (which already calls OV over HTTP).The endpoint was chosen for parity with
POST /sessions/{id}/extract.Design notes
retry_archive_extractionreuses_run_memory_extractionso all the existing telemetry, tracker integration, and error-handling paths apply unchanged. There is no duplicated Phase 2 logic.skip_previous_archive_check=Trueis mandatory on the retry path because the prior archive may itself have failed Phase 2 and left no.donemarker; without this flag, retry would deadlock waiting for that marker to appear..failed.jsonis written and the operator can call the endpoint once more.