Knowledge retention and central storage: copy what we do not control, and detect a vanished source first

Owner reasoning (2026-09-26), which specifies this precisely

original document stays in original place — do we need to retain artifact, after it is vectorised
if the source is gone we can't use it any more
if we want to retain some knowledge we have to copy to our controlled location — this is where central storage comes in

That resolves retention into a policy rather than a yes/no, and it scopes central storage to a retention tier, not a store-everything layer.

The policy this implies

Source	Who controls it	Treatment
Watch folder on our own fileserver	us	pointer + content hash — copying would duplicate and drift
Connector to third-party SaaS (Confluence, Notion, Slack, Jira, GDrive…)	them	pointer can die → copy to controlled storage
Crawled web page	nobody	will die → copy
Upload	no durable origin at all — the file was on someone's laptop	copy, or accept that re-extraction means re-upload

Copying everything is the store_authority rule-2 defect (a second original that drifts). Copying nothing loses the ability to ever re-extract. Copying what we do not control is the only position that is neither.

What is lost today when a source disappears — narrower than it sounds

The extracted text is already retained — that is the fact content, and it stays in the vector store and graph. What dies with the source is the ability to re-extract: better OCR later, fixed table parsing, a corrected extractor. So the exposure is "permanently stuck with the extraction quality we had on ingest day", not "the knowledge is gone".

That is a real cost — #13884 exists because scanned PDFs currently extract to nothing — but it is bounded, and it is what makes the copy tier worth scoping rather than skipping.

The prior gap: disappearance is not detectable

Nothing detects a vanished source. services/kb_folder_watcher.py has no deletion handling — one "removed successfully" string at :277, no on_delete, no unlink path. A deleted watched file, a deleted Confluence page and a 404'd URL all leave the fact in place pointing at a locator that no longer resolves, and no query can identify which facts have lost their evidence.

So the retention policy cannot currently be sized: nobody can measure how much of the KB has a dead source.

What exists to build on: nothing

No MinIO, S3-compatible store, NFS or shared volume in the fleet deployment.
integrations/cloud_integration.py is an outbound agent tool (list_s3_buckets against a user's own AWS), not storage infrastructure.
Redis and Postgres are already networked. ChromaDB supports both HttpClient (central) and PersistentClient (per-node disk), with the selection in knowledge/backends/async_chromadb_adapter.py:199 (Fix blocking ChromaDB operations in async functions #369) — but codebase_analytics runs its own PersistentClient concurrently with the KB's, so at least part of the vector store is node-local.

Acceptance criteria

Detect first. Source-liveness checking and watch-folder deletion handling, with a source_state on the fact. Cheap, and it makes the problem measurable.
Measure second. How many existing facts have an unresolvable source. That number decides how much retention is worth before anything is built.
Then storage. A node-shared, content-addressed store for retained copies — reachable from every fleet node, hash-keyed so the same document arriving twice is stored once. S3-compatible self-hosted (MinIO) is the obvious shape for this fleet; that is a recommendation, not a decision.
Retention is per source class, per the table above — not global, not per-file-opt-in.
The DOCUMENT entity from Umbrella: knowledge/memory is the relation system of record and every module a consumer — today nothing outside knowledge calls it #17526 carries one of two locator kinds: an external URI for sources we do not copy, or an internal object key for retained copies. One entity type, two resolutions.
Settle the deployment fact: is ChromaDB running central (HttpClient) or per-node (PersistentClient) in the fleet today? Readable only from the running system, not the repo. If per-node, a document ingested on one node is invisible to the others, which is a larger problem than retention.

Why this is ordered detect → measure → build

The retention tier is the expensive part and its size depends entirely on how often sources actually vanish. Building the store first means guessing that number. The detection work is small, independently useful, and turns the guess into a measurement.

Related: #17526 (DOCUMENT entity / provenance), #17527 (intake), #17537 (roadmap), #13884 (scanned PDFs extract to nothing — the concrete re-extraction case).

Correcting the storage recommendation — owner decision, and the precedent is already in this repo.

cifs or nfs · samba · same as connectors

The acceptance criteria above suggest "S3-compatible self-hosted (MinIO)". Disregard that. The mechanism is a mounted directory, exactly as the ingest side already works.

knowledge/connectors/file_server.py:8 states it:

Issue #1254: Ingests content from a mounted directory (NFS, SMB, local).

The connector does not speak SMB. It takes a Path, and the protocol is an infrastructure concern handled by the mount. Retention should be the same shape: AutoBot writes to a path; the fleet mounts that path via CIFS/Samba; the code stays protocol-agnostic and single-node local still works with no special case.

That is better than an object store here for concrete reasons, not just consistency:

No new dependency and no SDK — pathlib and hashlib, both already used by file_server.py.
One abstraction for both directions. Ingest reads a mounted dir, retention writes one. A reviewer learns it once.
Ansible already owns mounts, so "reachable from every node" becomes a deployment task rather than a service to run, monitor and back up.
Local development is unchanged — a local directory is a valid mount of one.

Revised criterion

~~S3-compatible object store~~ → a mounted directory, configured as a path, mounted CIFS/Samba across the fleet. Same pattern as file_server.py (Knowledge: Source Connector Framework (Backend Architecture) #1254). Single node uses a local directory with no code difference.

The one thing that must not be copied from the ingest side

file_server.py reads a mount. Retention writes to one, with content that arrived from outside — and that inverts the risk.

Retained copies must be content-addressed — named by hash, never by the user's filename. This repo has scar tissue precisely here: #17300 and #17302 (an unvalidated storage_path became an arbitrary-write primitive because pathlib discards everything left of an absolute segment), and today #17492, where a filename of fullwidth confusables survived a length bound, NFKC-normalised into ./../x, and collapsed two attachments onto one path.

Hash-naming removes that entire class rather than defending against it: no attacker-supplied component ever reaches the path, dedup comes free, and the same document arriving from two connectors is stored once. Keep the original filename in metadata where it belongs.

Two further notes for whoever builds it:

A share is not a database. Concurrent writes from multiple nodes need write-temp-then-rename within the same mount, which is atomic on both CIFS and NFS; writing in place is not.
UID mapping bites on CIFS. The mount's ownership semantics differ from local disk, and this fleet has already had a /var/log/redis vs /var/log/redis-stack ownership mismatch today (security(redis): the role's Redis Stack config is never read — requirepass is empty on a provisioned node, and one path has four spellings #17434). Pin the mount options in the ansible role rather than relying on defaults.

The detect → measure → build ordering above is unchanged: the mount does not need to exist before the liveness work, and the measurement is what sizes it.

Added requirement (owner, 2026-09-26):

each external knowledge document that we retained should be available to read in human-readable format

This is new surface — there is no document-serving precedent in the backend today. The only StreamingResponse uses are SSE and NDJSON (a2a.py, adapters.py); nothing returns a stored artifact, and knowledge_graph_routes.py:260's /documents/{id}/overview returns graph overview, not the file.

Three things follow, and two are not obvious.

1. Content-addressing and "readable" pull against each other — metadata resolves it

Hash-named files have no extension and no type. So the retained record must carry original_filename, content_type and size alongside the hash, and the serve route sets Content-Disposition and Content-Type from that metadata, never from the stored path. This is not a nuisance — it is the reason hash-naming is safe: the path contains nothing attacker-supplied, and the display name is data rather than a filesystem decision.

2. Serving retained documents can leak across the credential boundary — this is the sharp one

A retained copy was fetched with someone's credential. A Confluence page pulled through one user's OAuth is now a file on a share, and a naive serve route makes it readable by anyone with a knowledge-base account — including people who could not open it at the source.

Retention must not become privilege escalation. The copy inherits the access constraints of its origin, or the feature quietly flattens every source system's permissions into "anyone with AutoBot".

This is checkable and partly already solved: ConnectorCredentialStore delegates to the canonical secrets manager with accessed_by attribution, and the memory graph already declares SECRET, owns, shared_with, used_secret and performed_by (#608). So "which credential fetched this document" is expressible once DOCUMENT exists (#17526) — which makes the authorisation decision answerable rather than guessed. Unverified: whether used_secret is actually emitted; the same file records a case where uses_secret was written while used_secret was declared with no writer.

3. "Human readable" should mean two layers, not one

Serve the original artifact and the extracted text side by side. That makes extraction quality inspectable rather than assumed, which is exactly what #13884 needs — a scanned PDF currently extracts to nothing, and today the user cannot see that the indexed content is empty while the document plainly is not. Showing both turns a silent quality failure into an obvious one.

It also serves the hub goal directly: provenance you can open, not merely cite.

Acceptance criteria (added)

A retained document is retrievable in its original form, with Content-Type and filename resolved from metadata, not from the stored path.
The extracted text is viewable alongside it, so what was indexed can be compared with what the source contains.
Authorisation is inherited from the source, not granted by retention. A user who could not read the document at its origin cannot read the retained copy. Where the origin's permissions cannot be modelled, the document is restricted to the identity that ingested it rather than opened to all.
Served content cannot execute in the browser: Content-Disposition: attachment for anything not explicitly safe to render, and the existing _sanitize_html_content path reused for HTML rather than a second implementation. Retained artifacts are externally-authored bytes — HTML and SVG are active content.
A serving route exists for retained copies only. Documents we did not copy resolve to their original locator; the API must make clear which of the two a caller is getting, or a dead external link and a missing retained file become indistinguishable.

Clarification (owner, 2026-09-26) — and it corrects the bare-hash naming I proposed above:

human readable — can be opened with standard tools like text editors, document software — formats like md, docx etc

So the retained store must be browsable and openable by a person with a file manager and ordinary software, not only resolvable by the application. A file named a3f9c2e1… with no extension satisfies my earlier safety argument and fails this one completely — you cannot tell what it is, and nothing will open it.

Both properties are obtainable; they are not in conflict once the extension is treated as derived rather than supplied.

Revised naming

<hash>.<detected-extension> — for example a3f9c2e1….docx.

The hash keeps the safety property: no attacker-supplied component reaches the path, which is the whole point after security(llc): an authenticated user can write a file anywhere on disk via the attachment company_id query param #17300, security(llc): an agent-supplied storage_path is opened unvalidated, reading any text file on the host into the KB #17302 and security(llc): a confusable extension collapses the attachment basename, so two uploads overwrite each other #17492 (a confusable filename normalising to ./../x).
The extension is derived from the verified content type, not from the uploaded name. The precedent exists: verified_upload_extension(filename, file_content) — "the bytes outrank the name" (Verify ZIP-based office/ODF formats by content, not extension alone #16773) — and media/document/zip_formats.py already sniffs ZIP containers by their marker part rather than trusting the suffix.
Result: the file opens on a double-click, and the name still contains nothing a caller chose.

Retain two artifacts, not one

The original, in its own format — .docx, .pdf, .md, .xlsx — openable with the software it was authored in.
The extracted text, as .md or .txt, alongside it.

The second is the one that makes this durable. It is already produced at ingest (it is what gets vectorised), it costs nothing extra to write, and it means every retained document has a form that opens in any text editor forever — regardless of whether the original format is still supported, whether the proprietary reader still exists, or whether the extractor that produced it has since been replaced.

It also makes extraction quality directly inspectable: open the .docx and the .md side by side and the difference is visible. That is #13884's scanned-PDF case made obvious rather than silent — an empty .md next to a 40-page PDF needs no explanation.

What this does not mean

No format conversion at ingest. Do not normalise .docx to .md and discard the original — that would be re-extraction's whole purpose thrown away, and a lossy conversion is exactly what you cannot redo once the source is gone. Keep both; convert nothing.

Revised criteria

Retained files are named <content-hash>.<extension-derived-from-verified-content-type> — openable, with no caller-supplied path component.
Each retained document stores two files: the original in its own format, and the extracted text as .md/.txt.
The share is navigable by a human with a file manager: a person browsing the mount can find, identify and open a document without the application.
Original filename, source locator, ingest date and fetching identity live in metadata, and the serve route sets the download name from that — so a user receives quarterly-report.docx, not a3f9c2e1….docx.

Layout proposal (owner: "make the folder structure and naming so it is usable and can be navigated by human").

Why not the usual two-view store

The standard answer — a flat by-hash/ canonical store plus a by-source/ tree of symlinks — does not work here. This mount is CIFS/Samba, where symlinks require unix extensions that are commonly disabled, and hardlinks are unreliable across SMB implementations. A layout that depends on links will work in local development and break on the fleet.

So: one layout, navigable directly, with identity carried in the filename rather than by the path.

Proposed structure

<mount>/knowledge/
  <source-kind>/                 upload | watch-folder | web | confluence | jira | slack | …
    <source-instance>/           sanitised space key, folder alias, or domain
      <YYYY>/<MM>/
        <slug>__<hash12>.docx    the original, in its own format
        <slug>__<hash12>.md      the extracted text
        <slug>__<hash12>.json    metadata sidecar

Concretely:

knowledge/confluence/engineering/2026/09/quarterly-report__a3f9c2e1b407.docx
knowledge/confluence/engineering/2026/09/quarterly-report__a3f9c2e1b407.md
knowledge/confluence/engineering/2026/09/quarterly-report__a3f9c2e1b407.json
knowledge/upload/2026/09/scanned-invoice__7de4119a02c8.pdf
knowledge/upload/2026/09/scanned-invoice__7de4119a02c8.md

A person browsing the mount reads it without the application: where it came from → which instance → when → a recognisable name. Opening the .docx works; opening the .md works; the .json says what it is.

Why each part is shaped this way

<slug> — the original title, sanitised, so the file is recognisable. Derived from the source's own name, never used raw.
__<hash12> — first 12 hex of the content hash. Gives identity and dedup detection while keeping the name readable; the double underscore makes the boundary unambiguous when parsing back.
.<ext> — from the verified content type, not the supplied name (verified_upload_extension, "the bytes outrank the name", Verify ZIP-based office/ODF formats by content, not extension alone #16773).
<YYYY>/<MM> — bounds directory fan-out. A single directory with tens of thousands of entries is slow on SMB and unusable in a file manager.
.json sidecar — original filename, source locator, fetching identity, ingest timestamp, extractor version, content hash.

The sidecar is the durability argument

With it, the share is self-describing: if the knowledge database is lost or rebuilt, the mount still records what every file is, where it came from and when. Without it, a rebuilt index cannot re-associate documents with their provenance, and the copies become anonymous bytes — which is precisely the outcome retention exists to prevent.

CIFS constraints that must be handled in the sanitiser

These are not hypothetical; each one produces a real failure on a Samba mount:

Reserved device names — CON PRN AUX NUL COM1-9 LPT1-9 are unusable as components, with or without an extension. A Confluence space keyed AUX breaks the path.
Reserved characters — < > : " \ | ? * and control bytes. Colons are common in page titles.
Case-insensitivity — two sources differing only in case collide. Normalise case in the directory component or two instances silently share a folder.
Trailing dots and spaces are stripped by the server, so report .pdf and report.pdf may collide.
Path length — keep components short; the slug needs a cap (~64 chars) with the hash preserved, since truncating the hash instead would break identity.

Reuse rather than reimplement: transcriber/upload_security.py:77 generate_secure_filename, agent_loop/text_utils.py:16 slugify, and autobot_shared/security/path_validator.py (resolve_within_sandbox, validate_relative_path). Verify each against the CIFS list above before adopting it — they were written for local disk, and none of them was built with reserved device names in mind.

Dedup without links

Same content arriving twice — a file both watched and fetched by a connector — is detected by hash in the index, and the second ingest records a reference to the existing path rather than copying. The duplicate does not appear twice on the mount; the fact carries both source locators.

Acceptance criteria (added)

A person with a file manager can locate a document by source, instance and date without the application, and open it with ordinary software.
Every path component is derived and sanitised for CIFS specifically — reserved device names, reserved characters, case-collision, trailing dot/space, and a length cap that never truncates the hash.
Each retained document writes three files: original, extracted .md, and .json sidecar.
The mount is self-describing: the sidecars alone are sufficient to rebuild provenance if the index is lost.
A dedup test proves the same content from two sources stores one copy and records two locators.

Recording the seam for the watch-folder half, found while fixing the ingest path in #17022/#17531 — not building it here.

The gap is confirmed and narrower than "no delete handling": the handler has no delete event at all. KBFolderChangeHandler implements on_created (:112) and on_modified (:118) and nothing else. No on_deleted, no on_moved. So a watched file that is deleted or renamed leaves its fact in place, and the watcher never learns the source is gone — there is no code path that could mark it, not a path that marks it wrongly.

Two things now exist that make this cheap, which is the reason for this note.

First, the ingest metadata already carries the source path: build_watch_metadata in services/kb_watch_ingest.py writes file_path, filename and folder_id into every fact's metadata. A vanished source is therefore findable without a schema change — the join key is already stored.

Second, services/kb_watch_ingest.py is where the watcher's Knowledge Base calls live now, split out of the watcher so they can be exercised without standing up a filesystem observer. A mark_source_vanished(folder_id, file_path) beside ingest_watched_file is the natural shape, called from new on_deleted/on_moved handlers, and it gets the same testability for free.

One caution from the defect I was fixing. The watcher's KB write path had never executed — it called a method the KnowledgeBase does not define — and the reason nobody noticed for months is that the failure only ever incremented a bare errors counter. #17531 fixed that: failures now record what failed, and get_stats() surfaces last_errors per folder. Whatever marks a vanished source should report through the same place rather than adding a second silent counter, or detection will be as unobservable as the ingest was.

Worth deciding explicitly rather than by default: whether a vanished source marks the fact, tombstones it, or deletes it. Deleting is a data-removal decision an agent proposes and never takes (#17038), so marking is the only option that does not need an approval gate designed alongside it.

AC 6 answered from the running system: central server, and still not fleet-reachable

Settle the deployment fact: is ChromaDB running central (HttpClient) or per-node (PersistentClient) in the fleet today? Readable only from the running system, not the repo.

It is a central server — and it is bound to loopback only, so the consequence this AC warns about holds anyway, by a different mechanism than the one it anticipated.

Collected read-only from the deployment, nothing modified and no credentials used:

Observation	Evidence
Chroma runs as a service, not in-process	`autobot-chromadb.service` — loaded, active, running
It is genuinely Chroma, version 1.0.0	its own `/api/v2/heartbeat` and `/api/v2/version` answer on a loopback port; the v1 path returns Chroma's own "The v1 API is deprecated" message
26 collections in the default tenant/database	`autobot_code`, `autobot_docs`, plus per-project, per-company and per-work-item collections
The store is large and live	16 collection segment directories beside a 1.47 GB `chroma.sqlite3`, modified the same day as this reading
The `PersistentClient` directory is empty	the path the config names for per-node mode holds nothing
Bound to loopback, not `0.0.0.0`	the listener is on the loopback interface; every other data service on this host binds all interfaces

What that means for retention

The feared case was per-node PersistentClient. It is not that: one server owns the store, so two processes on this node share one view. But a loopback-bound server is unreachable from any other node, so another fleet node pointing an HttpClient at this host cannot connect — and would therefore need its own Chroma, with its own store. The outcome the AC names, "a document ingested on one node is invisible to the others", survives the answer.

So this AC is answered, and it hands #17538 a second question rather than closing one: is the intended topology one shared vector store, or one per node? Retention cannot be designed without that, because a content-addressed store "reachable from every fleet node" (AC 3) has no meaning if the index pointing into it is node-private.

Stated limits

One node. This is the host the deployment runs on; the AC says "in the fleet". Whether other nodes run their own Chroma is not answerable from here, and I did not reach out to them.
Loopback binding is what I observed, not what is configured. I did not read the deployment's config, deliberately — those files carry credentials, and the running state answers the question without them.
Nothing here required database access, which is why it is answerable at all. AC 2 (the fact-source census) does need it, and is a separate wall.

One observation outside this AC, for whoever owns it

Every other data service on this host binds all interfaces while Chroma binds loopback. That asymmetry is either deliberate or an oversight, and it is a question for the fleet-networking owner rather than for retention. Recorded here only because it came from the same reading; #17440/#17447 is the nearest prior art.

fix(kb): two overlapping sweeps can no longer undo each other (#17545)

AC 3 — "reachable from every fleet node": measured, and it is a gap on this node

Measured on this host just now with ss -ltn, service identified by the SSOT port (ssot_config.py:221 fixes ChromaDB at 8100):

Store	Port	Bind
ChromaDB	8100	`127.0.0.1` — loopback only
Redis	6379	`0.0.0.0`
Postgres	5432	`0.0.0.0` and `[::]`

A loopback-bound store is reachable from processes on that node and from nowhere else, so on the topology where the vector store is one shared authenticated store rather than a per-node copy, this bind cannot satisfy AC 3 — no second node can open the socket at all, with or without credentials.

The asymmetry is the part worth keeping: the two stores that are not the vector store already listen on all interfaces. So this is not a fleet-wide policy of keeping data stores local that Chroma happens to follow — Chroma is the outlier, and the direction of the fix is to bring it in line with Postgres and Redis, not to reconsider theirs.

Two limits on that measurement, stated rather than left to be assumed:

It is one node. "Reachable from every fleet node" is a fleet property; a single host's bind can show that this node's store is unreachable from elsewhere, and cannot show what the other nodes do. The fleet-wide half is unmeasured.
ss run without privileges does not print the owning process, so each row above is identified by port against the SSOT, not by inspecting the process. If some other process were holding 8100, the row would be mislabelled — the bind itself is what was measured.

A near-miss worth recording here rather than only in a message

While gathering this I initially took port 8000 for the vector store. It is not: a request to it returns a FastAPI 404 and the root path identifies it as the SLM Backend. Chroma is 8100, per the SSOT line above. Both happen to be loopback-bound, so a probe of the wrong port would have produced the right verdict for the wrong store — the kind of answer that is correct, complete, and about a different question, and that nothing downstream would have caught.

Anyone re-measuring this: identify the store by the SSOT port and confirm what answers there, rather than by the first port that looks like a service.

AC 3 — what the target configuration is, and the order it has to happen in

Now that the topology is decided as one shared authenticated store, the gap is definable. Measured on this node; the deployed tree read a4768fd323f5 both before and after every observation below, so this is one result about one SHA. (That SHA was origin/main at measurement time and is already behind it — parity does not survive the queue.)

Current posture, measured

Question	Observation
Where does the vector store listen?	`127.0.0.1:8100` — loopback only (`ss -ltn`; port from `ssot_config.py:221`)
Where do the other two stores listen?	Postgres `0.0.0.0` + `[::]`, Redis `0.0.0.0`
Does it require credentials?	a request carrying no credentials to `/api/v2/heartbeat` and `/api/v2/version` returns HTTP 200 (`/api/v1/heartbeat` returns 410 — v1 is gone, v2 is the live API)

So today the store is unreachable from other nodes and unauthenticated to anything on this one. Its only protection is the network boundary.

What that means for the fix, and why the order matters

The target moves the store off the boundary that is currently protecting it. Opening the bind first would turn a local-only unauthenticated store into a fleet-reachable unauthenticated store — strictly worse than today, and reachable by a single config change made in good faith.

So the criterion should be sequenced rather than stated as one outcome:

Auth on, verified by a refusal. AUTOBOT_CHROMADB_AUTH_TOKEN already exists (ssot_config.py:1604) and feeds CHROMA_SERVER_AUTHN_* through utils/chromadb_auth.chroma_client_auth_kwargs, which every client path already calls (async_chromadb_client.py:527, chromadb_client.py:39, cli/doctor.py:74). The SSOT default is the empty string, documented as auth disabled. Evidence for this step is a credential-free request that is refused, not a configuration file that mentions the variable.
Then the bind widens, from the SSOT host knob (AUTOBOT_CHROMADB_HOST, ssot_config.py:189, default 127.0.0.1) — never a hardcoded address.
Then reachability is demonstrated from a second node, which is the part no single-host measurement can establish.

Two things I did not establish

Whether the token is set in the deployment. The variable name appears in host env files, which establishes nothing about whether it holds a value — and I did not read one. A name match is not a configuration. The behavioural answer above (HTTP 200 without credentials) is consistent with auth being disabled, and that is the evidence I would act on.
Whether the data endpoints enforce auth even though heartbeat and version do not. Chroma's token authn normally covers every endpoint, so a 200 on heartbeat is good evidence, but the discriminator would be a credential-free collection listing and I deliberately did not run one — establishing an auth posture does not require reading the knowledge base.

Suggested criterion wording

The vector store refuses a credential-free request (evidence: the refusal, from a client that sends nothing)
Its bind address comes from AUTOBOT_CHROMADB_HOST and is not loopback in the fleet deployment (evidence: the listening socket, plus the SSOT read — no hardcoded address anywhere)
A second fleet node completes a round trip to it (evidence from that node, not from this one)
The three steps land in that order, and a review of the change says so — widening the bind before auth is enforced is the one sequence that is worse than doing nothing

feat(kb): detect whether a fact's source still resolves (#17545) (#17615

AC 3 gap settled: the vector store's data endpoints answer unauthenticated too

The open question was whether authentication is enforced on the data endpoints while the liveness
endpoint is open. It is not. Measured on the deployment just now.

Method chosen deliberately: status codes only, response bodies discarded (curl -o /dev/null). No
collection was listed, no name read, no content touched. The question is about configuration and it is
answered without reading the knowledge base to settle it.

Endpoint	Credential-free result
liveness (v2)	200
collections listing (v2, default tenant/database)	200
liveness (v1)	410 — retired API
a v1 data path	410 — retired API

So the store is unauthenticated at the data layer, not merely at its health check. Its only protection
is the loopback bind.

What this settles, and what it does not

Settles: the sequencing constraint recorded earlier is confirmed rather than assumed. Authentication
must be in place before the bind widens, demonstrated by a refusal from an unauthenticated
caller. Widening first would expose a credential-free data layer to every node, which is strictly worse
than today's posture.

Does not settle: whether writes are equally open. Only reads were probed, and a write probe would
mutate state, so it is not an appropriate way to answer a configuration question. Assume writes are open
unless a refusal is demonstrated — that is the safe direction and it does not change the plan.

Cross-reference

A sibling store is in a worse position and already has a home at #17500: unauthenticated, no
password configured, and reachable beyond loopback. The class is the same — reachable implies
authenticated — and the ordering rule applies to both: credentials land before reachability widens,
never after. The vector store is the safer of the two today only because of where it sits, which is
precisely what the scaling work removes.

Classification (for work assignment — not a fix proposal)

Scope: kb (secondary chromadb)
Primary files/dirs:
- autobot-backend/knowledge/source_liveness.py and autobot-backend/models/knowledge_fact.py (verified on main; the detect half, via feat(kb): no fact records whether its source still resolves — probe and measure before retention is scoped #17545)
- autobot-backend/services/kb_folder_watcher.py and autobot-backend/services/kb_watch_ingest.py (verified on main; the watch-deletion half)
- autobot-backend/knowledge/backends/async_chromadb_adapter.py (verified on main)
- a retained-copy store and serve route (inferred; the issue says these do not exist yet)
Umbrella: yes, and it is a container. Children:
AC3-5 (storage, per-class retention, the DOCUMENT locator) have no child issue. The children are the work.
Blocked by:
- AC2 (measure): Source-liveness sweep reaches one ingest leg of five; the upload leg has a locator and is excluded #17637 (the sweep reaches one of five ingest legs) and a database read, which the comments call "a separate wall".
- AC3 (storage / shared store): auth on the vector store before it is bound beyond loopback (comments 7-8), and evidence from a second node.
- bug(kb): the watch folder never notices a deleted or renamed file — no on_deleted, no on_moved, and change_type is decorative #17546: was blocked by bug(kb): the watch folder ingested nothing — KnowledgeBase has no add_fact, and #13551's fix raised the identical error #17531, which is now closed.
- Already ruled: the owner chose a mounted CIFS/Samba directory, not an object store (comment 1).
Read vs inferred: Read the issue body, its 9 comments and its sub-issue list, and verified the listed paths on main. AC6 is ticked from host evidence in comment 6.
Undetermined: whether the DOCUMENT entity (Umbrella: knowledge/memory is the relation system of record and every module a consumer — today nothing outside knowledge calls it #17526/feat(memory): ENTITY_TYPES has no artifact types — add DOCUMENT and CODE so provenance is expressible #17539) exists yet, and which nodes besides the measured one run a vector store.
Same-file note: services/kb_folder_watcher.py is also the primary file of bug(kb): the watch folder never notices a deleted or renamed file — no on_deleted, no on_moved, and change_type is decorative #17546, bug(kb): watch-folder modifications are gated on a per-process dict, so a file that predates the process is never re-ingested at all #17547 and Connectors must declare what they need, and test_connection must return why it failed #17554. Under the same-file rule, work on it goes to one session.

Generated by Claude Code

Chunking triage — a proposal, not an assignment

Proposed priority: none. A container is not work; see the umbrella line below.
triage(v0.9.0): release criticality of the 123 open issues — 26 blocking, 36 small, 18 umbrellas, 41 misfiled, 2 undetermined #17639 criterion met: not applicable — a tracker cannot itself be a shipped breakage.
triage(v0.9.0): release criticality of the 123 open issues — 26 blocking, 36 small, 18 umbrellas, 41 misfiled, 2 undetermined #17639 bucket: 3 — umbrella, research or verification-only tracker
Umbrella / container: YES — triage(v0.9.0): release criticality of the 123 open issues — 26 blocking, 36 small, 18 umbrellas, 41 misfiled, 2 undetermined #17639 records it as a tracker with unfinished children: children-open feat(kb): no fact records whether its source still resolves — probe and measure before retention is scoped #17545 bug(kb): the watch folder never notices a deleted or renamed file — no on_deleted, no on_moved, and change_type is decorative #17546 bug(kb): watch-folder modifications are gated on a per-process dict, so a file that predates the process is never re-ingested at all #17547 (verified via get_sub_issues, 0/3 done); measure/storage ACs need live census and a d. It must not sit in a numbered chunk.
Pre-filter: clean — no merged commit on origin/main since 2026-08-14 references this issue.
Basis: triage(v0.9.0): release criticality of the 123 open issues — 26 blocking, 36 small, 18 umbrellas, 41 misfiled, 2 undetermined #17639's per-issue row, reused rather than re-derived — "children-open feat(kb): no fact records whether its source still resolves — probe and measure before retention is scoped #17545 bug(kb): the watch folder never notices a deleted or renamed file — no on_deleted, no on_moved, and change_type is decorative #17546 bug(kb): watch-folder modifications are gated on a per-process dict, so a file that predates the process is never re-ingested at all #17547 (verified via get_sub_issues, 0/3 done); measure/storage ACs need live census and a design that will not close by code today"

Nothing was relabelled, moved or closed by this pass.

Do not close this on the child count: three children are closed and five of its own six criteria are not

All three native sub-issues are CLOSED, which is the first thing a reader checks. Five of the six criteria in this body are unticked, and none of them is delegated to a child — they are this umbrella's own work:

Detect first — source-liveness checking and watch-folder deletion handling, with a source_state on the fact.
Measure second — how many existing facts have an unresolvable source. The number that decides how much retention is worth.
Then storage — a node-shared, content-addressed store for retained copies, hash-keyed so the same document arriving twice is stored once.
Retention per source class, per this issue's own table — not global, not per-file opt-in.
The DOCUMENT entity from Umbrella: knowledge/memory is the relation system of record and every module a consumer — today nothing outside knowledge calls it #17526 carries one of two locator kinds — an external URI for sources we do not copy, or an internal object key for retained copies.

The sixth (settle the deployment fact: central HttpClient or per-node PersistentClient) is ticked, and it is the one that was readable only from the running system.

So the child count and the criteria disagree, and the criteria are the closure argument. Closing on "3/3 children closed" would close five criteria nobody has done — including the storage decision, which is the largest single piece of work named here.

Recorded during a read-only audit of the 13 open umbrellas in v0.9-umbrellas; nothing here was ticked or changed.

Knowledge retention and central storage: copy what we do not control, and detect a vanished source first #17538

Description

Owner reasoning (2026-09-26), which specifies this precisely

The policy this implies

What is lost today when a source disappears — narrower than it sounds

The prior gap: disappearance is not detectable

What exists to build on: nothing

Acceptance criteria

Why this is ordered detect → measure → build

Activity

mrveiss commented on Sep 26, 2026

Revised criterion

The one thing that must not be copied from the ingest side

mrveiss commented on Sep 26, 2026

1. Content-addressing and "readable" pull against each other — metadata resolves it

2. Serving retained documents can leak across the credential boundary — this is the sharp one

3. "Human readable" should mean two layers, not one

Acceptance criteria (added)

mrveiss commented on Sep 26, 2026

Revised naming

Retain two artifacts, not one

What this does not mean

Revised criteria

mrveiss commented on Sep 26, 2026

Why not the usual two-view store

Proposed structure

Why each part is shaped this way

The sidecar is the durability argument

CIFS constraints that must be handled in the sanitiser

Dedup without links

Acceptance criteria (added)

mrveiss commented on Sep 26, 2026

mrveiss commented on Sep 27, 2026

AC 6 answered from the running system: central server, and still not fleet-reachable

What that means for retention

Stated limits

One observation outside this AC, for whoever owns it

mrveiss commented on Sep 27, 2026

AC 3 — "reachable from every fleet node": measured, and it is a gap on this node

A near-miss worth recording here rather than only in a message

mrveiss commented on Sep 27, 2026

AC 3 — what the target configuration is, and the order it has to happen in

Current posture, measured

What that means for the fix, and why the order matters

Two things I did not establish

Suggested criterion wording

mrveiss commented on Sep 27, 2026

AC 3 gap settled: the vector store's data endpoints answer unauthenticated too

What this settles, and what it does not

Cross-reference

mrveiss commented on Sep 28, 2026

Classification (for work assignment — not a fix proposal)

mrveiss commented on Sep 28, 2026

Chunking triage — a proposal, not an assignment

mrveiss commented on Sep 28, 2026

Do not close this on the child count: three children are closed and five of its own six criteria are not

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions