Skip to content

Knowledgebase database generated at build time instead of imported at startup #335

Description

@DutchJaFO

Background

The Knowledgebase ships with the changelog's proven mechanics: bundled JSON imported into a database at
startup, with a file fallback when the database is unavailable. That works, and it is what let the
Knowledgebase release without waiting on new machinery.

It does mean the content is imported on every startup, and that in the case the Knowledgebase exists
for — a degraded instance with no usable database — it is always served from the fallback rather than
the store. Generating the database once, after the build, and shipping it inside the image removes
both: no startup import, and a store that is available regardless of what the domain database is doing,
because it is a different file on a different mount.

This is an optimisation of something already working, deliberately separated so no release depends on
it.

Depends on #333 for the schema, content files and reader.

The constraint that decides the design

CLAUDE.md records that the Home Assistant supervisor mounts /app read-only — that is why Serilog
is configured programmatically rather than via ReadFrom.Configuration, whose DllScanningAssemblyFinder
throws there.

So a database baked into the image is opened from a read-only directory, which is the case SQLite
cannot serve when it must create the -shm wal-index: a database left in WAL mode by its generator
will fail to open, in the add-on, with SQLite Error 14: 'unable to open database file'. This is the
same failure #326 investigated from the other direction.

The generated file must therefore be checkpointed out of WAL mode, and the reader must open it
read-only explicitly. Verifying this needs a genuinely read-only mount — a local run on a writable
filesystem would pass while the add-on failed, which is exactly how #294's incident stayed hidden.

What needs to be done

  1. A generator that builds the Knowledgebase database from the per-language JSON sources, run after
    the application build. Per ADR 010 this is a dotnet-script .csx under scripts/ or a project
    under tools/ — never a shell or Python script.
  2. The generated database is checkpointed and not left in WAL mode, so it can be opened from a
    read-only directory. Assert this about the artefact rather than assuming the generator's defaults.
  3. The reader opens it read-only, and reads it without any dependency on the data volume or the
    domain database.
  4. The Dockerfile produces the database and includes it in the image, and the build fails if
    generation fails — a silently missing Knowledgebase would degrade to the file fallback without
    anything noticing, which is the failure mode Move changelog content to database-backed System_Changelog table #309 hit when its database went away mid-run and the
    JSON fallback covered for it silently.
  5. The startup import is removed once the built artefact is the source, along with its readiness
    signalling. The file fallback stays: it is what covers a corrupted or missing artefact.
  6. CI asserts the database is present and non-empty in the publish output, alongside the existing
    data/sources/ assertion.

Expected tests

Test class Test method Starts
KnowledgebaseArtefactTests GeneratedDatabase_IsNotInWalMode ❌
KnowledgebaseArtefactTests GeneratedDatabase_ContainsEveryEntryFromEveryLanguage ❌
KnowledgebaseArtefactTests GeneratedDatabase_OpensReadOnly ❌
KnowledgebaseReaderTests GetEntry_ReadsFromTheBuiltArtefactWithoutTheDataVolume ❌
KnowledgebaseReaderTests GetEntry_ArtefactMissing_FallsBackToTheBundledFiles ❌

A T2 pass is required and is the only honest verification of the central constraint: a container run
with the image's own directory read-only, proving the artefact opens where the HA add-on will actually
open it. T3 confirms it in the live supervisor.

Definition of done

  • All expected tests listed above start red before implementation
  • All requirements implemented
  • All expected tests pass (green)
  • No regression in related tests
  • Findings summarised in a closing comment

Activity

  1. added this to the v1.9.0 milestone on Aug 20, 2026
  2. added a commit that references this issue on Aug 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions