Skip to content

unstructured2graph: make GLiNER2 the default extraction backend across the project - #444

Merged
antejavor merged 3 commits into
mainfrom
feat/gliner2-default
Oct 7, 2026
Merged

antejavor merged 3 commits into
mainfrom
feat/gliner2-default

Conversation

@antejavor

Copy link
Copy Markdown
Contributor

Stacked on #441 (it needs hygm.default_model()). Once #441 merges, retarget this PR to main before merging it, or it lands on the dead branch.

GLiNER2 becomes the default extraction backend everywhere. LightRAG is kept as an override. This is the extraction side of map #431: learned ontologies are derived and applied through GLiNER2.

Changes

  • unstructured2graph declares gliner2[local]>=2.0.0 as a dependency. The manual-install error path and the ty: ignore suppressions on gliner2 imports are removed, and JointIEConfig now gets explicit arguments.
  • from_documents(..., ontology=Ontology) enforces an in-memory ontology, which takes precedence over ontology_path. Learned models live in the graph, not in a file.
  • sessions-graph reconcile_session:
    • Defaults to GLiNER2Backend over hygm.default_model(), built once per SessionsGraph, since loading the model is the slow part.
    • With enforce_ontology, checks labels against the same vocabulary it extracted with, unless an ontology_path overrides it.
    • LightRAG still writes the session summary.
    • The reconciliation extra now declares hygm.
  • context-graph-eval: --extraction-backend defaults to gliner2 (over the LongMemEval vocabulary). Saved runs from before the field existed still read as lightrag.
  • Docs: README/AGENTS updated. GLiNER2 is shown first; LightRAG is the alternative.

Security trade-off (deliberate)

gliner2[local] pins transformers<5, which can't share the workspace lock with the transformers>=5.0.0rc3 floor for CVE-2026-1839 (an RCE in Trainer checkpoint loading). The floor is dropped, and transformers resolves to 4.57.6. No code in the repo imports transformers or uses Trainer. Restoring the floor is tracked in #443.

Tests

Run against a real Memgraph:

Suite Result
sessions-graph 95 passed
real end-to-end reconcile (GLiNER2 extraction, LLM summary) passed
unstructured2graph 151 passed, including the real GLiNER2 end-to-end tests
context-graph-eval 187 passed

The reconciliation tests build the real default backend; only the model checkpoint is stood in for. ruff and the full-workspace ty check . are clean.

…s the project

- unstructured2graph declares gliner2[local] as a dependency; the
  manual-install fallbacks and import suppressions go.
- sessions-graph reconcile_session defaults to GLiNER2 over
  hygm.default_model(), built once per SessionsGraph, and enforces the
  vocabulary it extracted against. LightRAG stays available as an override
  and still writes the session summary.
- from_documents takes an in-memory ontology, for models held in the graph.
- context-graph-eval defaults --extraction-backend to gliner2.

gliner2[local] pins transformers<5, so the workspace's CVE-2026-1839 floor
is dropped for now; restoring it is tracked in #443.
@antejavor
antejavor force-pushed the feat/gliner2-default branch from 0ea048c to 4354e48 Compare October 7, 2026 13:02
@antejavor
antejavor changed the base branch from feat/431-hygm-default-ontology to main October 7, 2026 13:02
@antejavor
antejavor force-pushed the feat/gliner2-default branch from 0ea048c to 4354e48 Compare October 7, 2026 13:02
@antejavor antejavor closed this Oct 7, 2026
@antejavor antejavor reopened this Oct 7, 2026
@antejavor
antejavor merged commit edc49ff into main Oct 7, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant