fix: preserve knowledge base document IDs during backup restore

[dafyy321-pixel]

Fixes #10529.

This preserves the integer chunk IDs when restoring a knowledge-base backup, so the restored SQLite rows continue to match the IDs in the copied FAISS index.

A deleted chunk leaves a gap in SQLite's primary-key sequence. For example, a valid knowledge base can contain IDs 1 and 3. The backup exports those IDs along with the existing index, but the importer currently inserts the text rows with automatic allocation. The restored rows become 1 and 2, while FAISS still contains vectors labelled 1 and 3. The restore can finish normally even though dense retrieval can no longer resolve one of its vectors to the original text.

The fix uses the identity already present in the backup. It requires no archive-format change, index rebuild, or additional embedding request.

Modifications

Preserve the integer primary key during restore

astrbot/core/db/vec_db/faiss_impl/document_storage.py adds one optional, keyword-only argument to insert_document():

integer_id: int | None = None

The supplied value is used as Document.id. With the default None, SQLite still allocates the integer ID automatically, as it does for ordinary inserts today. The existing positional arguments and return value remain the same.

astrbot/core/backup/importer.py passes integer_id=doc.get("id") from the exported document entry. The comment beside that argument explains why it is needed: the FAISS file being copied into the restored knowledge base still refers to those integers.

The string doc_id, text, and metadata retain their existing import paths. The distinction between doc_id and id is important here. Preserving the string identifier does not repair a vector that points to a different SQLite primary key.

Keep normal allocation and older document entries working

The explicit ID is limited to callers that provide it. New documents continue to use automatic allocation, and an imported entry without an id follows the same path because doc.get("id") returns None.

For a restored database containing IDs 1 and 3, the next new document receives ID 4. That keeps new inserts above the restored IDs instead of reusing an integer that the copied index already associates with an older chunk.

Add a restore regression using real storage

tests/unit/test_kb_backup_restore.py adds two cases. The main case creates real SQLite and FAISS stores and uses a small, deterministic embedding provider. It inserts alpha, bravo, and charlie, deletes bravo, then restores the surviving document entries in reverse order and copies the original FAISS index into the restore destination.

Reversing the entries is deliberate. It verifies that restoration preserves the supplied identity even when allocation order would otherwise assign a different integer to the same text.

The assertions cover:

Check	Verified result
Surviving source IDs	alpha: 1, charlie: 3
Restored IDs after reverse-order import	alpha: 1, charlie: 3
Dense query for charlie	Returns the charlie row
FTS lookup for charlie	Returns integer ID 3 with the matching string ID
New insert after restore	Allocates integer ID 4
Query after the new insert	Still returns the original charlie chunk
Close and reopen	Both the restored charlie chunk and newly inserted delta remain retrievable

The second case imports a document entry without id, checks automatic allocation of ID 1, then verifies that an ordinary insert receives ID 2.

Behavior after the change

In the sparse-ID example, both stores retain the same relationship:

SQLite documents: alpha -> 1, charlie -> 3
FAISS vector IDs: alpha -> 1, charlie -> 3
Next inserted document: delta -> 4

Dense lookup no longer loses the row for ID 3, and inserting a new document does not cause the old vector to resolve to that document's text. FTS and dense retrieval agree on the restored chunk's identity.

Verification Steps and Test Results

Validated locally on Windows with Python 3.12.14 and Ruff 0.15.22.

The relevant test selection was:

python -m pytest tests/unit/test_kb_backup_restore.py tests/unit/test_document_storage_fts.py tests/test_backup.py -q
82 passed

This includes the two new restore cases, the existing FTS document-storage coverage, and the backup suite.

Formatting and static analysis also passed:

ruff format --check .
516 files already formatted

ruff check .
All checks passed!

ruff format --check --no-respect-gitignore tests/unit/test_kb_backup_restore.py
1 file already formatted

The explicit check of the new test file ensures that it is covered even when repository ignore rules affect the full-directory format command. git diff --check is clean.

Compatibility and review scope

This changes two production files and adds one test file. The optional argument preserves existing calls to insert_document(). The archive schema, database schema, backup routes, response envelopes, and frontend remain the same.

The regression verifies the restore relationship at the document-store/index boundary rather than relying on a successful import message or matching entry counts. There is no UI navigation or label change, so the WebUI instructions and screenshots do not need an update for this fix.

The change can be reviewed independently of selective-backup work such as #10151. If the import loop is later converted to batch insertion, the same requirement still applies: the integer IDs supplied by the backup must be retained when the existing FAISS index is restored alongside them.

Checklist

• This is a bug fix with no new feature proposal.
• This is not a breaking change; existing insert callers retain automatic allocation.
• Verification steps and passing results are included above.
• Checked the WebUI documentation requirement; this change does not alter navigation, page structure, or labels.
• No new dependencies or lockfile changes.
• The changes are limited to document-ID restoration and its regression coverage.

Summary by Sourcery

Preserve knowledge-base document identities during backup restoration to keep SQLite records aligned with their existing FAISS vector IDs.

Bug Fixes:

• Preserve integer document IDs when restoring knowledge-base backups so copied FAISS indexes continue resolving vectors to the correct SQLite documents.

Enhancements:

• Retain automatic ID allocation for new documents and restore entries without an explicit integer ID.

Tests:

• Add integration-style regression coverage for sparse-ID backup restoration, dense and FTS retrieval, subsequent inserts, reopening, and legacy entries without integer IDs.

[sourcery-ai]

Hey - I've reviewed your changes and they look great!

Sourcery assessment

Needs a human reviewer. If the restored integer IDs are wrong or collide, the copied FAISS index can retrieve the wrong knowledge-base documents and the resulting database state survives a code revert. The damage is bounded and can be repaired by restoring the backup again after correcting the import.

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

[dafyy321-pixel]

The pull-request workflows are marked action_required, and none of their jobs has started. Could a maintainer approve the fork workflow runs when reviewing this PR? The same approval state currently applies to #10532 and #10534.

Local verification for this backup fix is complete: the restore, FTS, and backup tests pass (82 tests), along with Ruff formatting and lint checks. The PR description includes the exact commands and the checks for sparse IDs, reverse-order restoration, a subsequent insert, and retrieval after reopening the stores.