Skip to content

A new base starts with no ontology pack #580

Closed

Description

@WaylandYang

Today a base starts with schema.org unless someone unticks it: the creation dialog pre-selects it (DEFAULT_ONTOLOGY_PACKS in web/src/api.ts, 0008), and the General base made at registration installs it without a dialog (DEFAULT_PACK in api/kbs.rs, #322). The proposal is that a new base starts with no ontology at all; packs stay available and remain the cold start for anyone who picks one.

Why

  • schema.org is 800 classes. On a base that has nothing else, it is the whole prompt: The base created at registration installs an ontology no local model can hold #519 measured 105k tokens per chunk once retrieval fell back to the full ontology, and every local 32k endpoint answered 400. The fix there made retrieval run; it did not make the pack a good default for a base whose documents are about something else.
  • The Chinese-label question in 0008 has been open since the record and got worse at the 2026-09-02 check: a Chinese base starts with 800 English classes and a handful of aligned labels.
  • What The base created at registration installs no ontology pack #322 wanted from the pack, domain and range so a fact has a direction, now has other sources: the ontology grows from the documents under 0003 / 0009 / 0010 with auto_extend_ontology on by default, and 0036 has exploration propose the classes a data source needs. An empty start means the first documents shape the ontology instead of a vocabulary the base may never use.

What changes

  1. Registration installs no pack. The General base is empty until its owner picks one or the first extraction proposes classes.
  2. The creation dialog pre-selects nothing. The pack list stays, with the same summaries and counts.
  3. 0008 gets a revision note: packs are a cold start someone chooses, not the default; The base created at registration installs no ontology pack #322 is reversed with the reason.
  4. Nothing is removed from existing bases.

To measure before deciding it is right

The reason #322 exists: with no classes and no properties, does extraction on a fresh base still produce directed facts, or does it fall back to related_to? Run the extraction bench (scripts/bench/run.mjs, ai-timeline corpus) on a base with no pack against the same base with schema.org, and compare direction rate, related_to share, and the class count the base ends with. That number decides whether cut 1 ships as is or needs the cold-start proposals of 0036 first.

Where this came from: the observation that the default base should not carry a built-in ontology (2026-09-10), against #519's earlier conclusion to keep the pack and fix retrieval. Both are on record; this issue is the place to settle it with a number.

Activity

  1. WaylandYang commented on Sep 10, 2026

    @WaylandYang
    ContributorAuthor

    Measured. Two fresh bases on the same server (dev at 0fc44a7e), the same 15 ai-timeline documents (276k chars), DeepSeek-V3.2 through SiliconFlow, auto_extend_ontology on in both (the product default), bootstrap_ontology allowed to finish before reading. One with no pack, one with schema.org installed at creation.

    no pack schema.org
    ontology in each extraction prompt none; the whole call is ~2k tokens (median 1988, p90 2389) the retrieved slice for that chunk (40 classes, 30 relations, 120 domain relations, 40 attributes): the whole call is 10–37k tokens (median 17957, p90 22834)
    ontology after bootstrap 13 classes, 175 relations, all in the corpus's own words 915 classes, 1671 relations (+112 minted)
    facts 3780 3760
    edges (entity → entity) with a predicate 1132 / 2007 (56%) 1059 / 2147 (49%)
    facts without a predicate (edges + attributes) 70% 65%
    entities typed 1723 / 2160 (80%) 1868 / 1902 (98%)
    type accuracy, Wikidata answer sheet, entities present 367 / 479 (77%) 419 / 468 (90%)
    truth entities not found in the base 154 167
    extraction drops 346 876 (342 domain_mismatch, 93 direction_corrected)
    extraction time, two runs in parallel 139 min 143 min

    Reading it:

    • Predicates do not come from the pack. With schema.org installed, four facts in five still had no predicate before bootstrap (the corpus says released, acquired, announced; the pack has employee, founder, funder), and 342 facts were dropped for domain_mismatch. After bootstrap the empty base has more directed edges than the pack base, because every relation it minted is a phrase the corpus actually uses.
    • Types do come from the pack. 98% typed at 90% accuracy against 80% typed at 77%. Part of the gap is vocabulary rather than judgment: the answer sheet is written in schema.org keys, and the empty base minted location, company, software, technology where the sheet wants place, corporation, software_application; those account for 32 of the 89 wrong-class misses. Counting them as right would put the empty base near 83%. The other part is real: 437 entities left untyped, against 34.
    • What the empty base cannot do: without domain/range there is no direction check, so the 93 direction_corrected drops on the pack base have no counterpart here; some of those edges are presumably backwards in the empty base and this run cannot count them. That is the The base created at registration installs no ontology pack #322 concern, and it stays open as a number to get.
    • Bootstrap is the cold start, and it is one shot after extraction is idle. Its first attempt died on a SiliconFlow connection error and the retry carried it; a base with a single document never crosses MIN_DOCS = 2, so a General base with one file uploaded stays untyped until the second.

    Where that leaves the proposal: dropping the default pack costs about ten points of type accuracy on this corpus and 20% of entities untyped, and buys extraction calls of ~2k tokens instead of ~18k, an ontology of 13 classes in the base's own vocabulary, and more edges with a predicate. The pack stays one click away for a base that wants schema.org's vocabulary, and the number to watch after the change is the untyped share on real bases.

    Single run each; the extraction bench's variance is about ±3 entities on 52, so treat differences under ten as noise. Bases 01a08921-81e5… (no pack) and 01a08921-d055… (schema.org) on the bench database, if anyone wants to look.

    Correction (2026-09-10): the first version of this comment said the schema.org base put ~105k tokens of ontology into the extraction prompt. That is the size of the full class list (497k chars against a 24k budget), which the bench reports as a proxy for ontology scale; since #523 a base over budget retrieves a slice per chunk, and the server's own usage log for these two runs is what the table now shows. The tokens come from llm usage lines, attributed by size: the 593 calls under 3k are the no-pack chunks and the small adjudication calls, the 383 calls over 10k are the pack base's chunks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions