Skip to content

Repository files navigation

org-semantic

Search a tree of org-mode notes by meaning or by words. One static binary, no database, no Python. It runs as a one-shot command, or stays resident for Emacs — over a pipe, never a port.

Full documentation

The screenshot and the worked example below search Daniel Bias's braindump, someone else's public vault of 753 org notes in English and Italian, cloned into braindump/. So you can run them as they stand.

From Emacs

The org-semantic results buffer, showing an English question answered by Italian notes and English ones ranked together

M-x org-semantic-find searches the vault the current buffer belongs to. The question is in English, the note answering it is in Italian, and an English note is ranked beside it.

RET goes to the line under point, n and p walk the passages, k and + widen the list or deepen it, g asks again. It is a next-error client, so M-g M-n walks the hits from anywhere.

f is the one to know: follow mode opens each passage in its own note as point reaches it, without taking point out of the list — so n and p read the vault rather than an index of it. It is off until you press f, or until the :hook below turns it on for every results buffer.

Setting it up

The package is in lisp/. There are no default global bindings and there will not be — C-c and a plain letter is yours rather than a package's — so a recommendation is as far as this goes:

(use-package org-semantic-results
  :load-path "/path/to/org-semantic/lisp"
  :custom (org-semantic-vault-root "~/notes")
  :bind (("C-c n s" . org-semantic-find)
         ("C-c n S" . org-semantic-find-at-point)
         ("C-c n R" . org-semantic-reindex))
  ;; Follow mode, on for every results buffer.
  :hook (org-semantic-results-mode . next-error-follow-minor-mode)
  ;; Reindex a vault as its notes are saved.
  :init (org-semantic-auto-reindex-mode 1))

org-semantic-vault-root is the one setting that has to be right: it says which directory your notes are, and every buffer that says nothing else — *scratch*, the agenda — searches it. With several vaults, leave it nil and let each one declare itself in its own .dir-locals.el.

There is no org-semantic-executable here because there need not be: a binary in org-semantic-install-directory is found on its own. Set it only to name one somewhere else. Searching from Emacs covers the rest, including every key the results buffer takes.

From the CLI

org-semantic index  ~/notes --both      # build both indexes
org-semantic search ~/notes "how did we decide to do it that way"  # by meaning
org-semantic search ~/notes 'tag:meeting budget' --lexical         # by word
org-semantic serve                      # JSON-RPC over stdio, for an editor

org-semantic -h explains every command; how a vault is indexed — languages, excluded subtrees, what happens to src blocks — lives in a JSON policy file, starting from config.example.json.

One run in full, over the vault named above:

$ org-semantic index braindump/roam --both --model e5-small
  20200924090307-elementi_di_probabilita_e_statistica.org: could not be read, so it is not indexed: stream did not contain valid UTF-8
753 org files
  256 sections were divided to fit the 350-token budget
3038 chunks · 3038 to embed · scanned in 1.5s
model loaded in 0.9s
embedded 3038 chunks in 77.8s (39/s)
wrote braindump/roam/.org-semantic/semantic/e5-small (4.7 MB of vectors) in 80.3s total
  20200924090307-elementi_di_probabilita_e_statistica.org: could not be read, so it is not indexed: stream did not contain valid UTF-8
753 org files
lexical index: 2863 chunks written in 0.4s

$ org-semantic search braindump/roam "what happens when a process is scheduled off the cpu" 2 --per-file 2

0.860 (+1.7σ)  Sistemi Operativi > Gestione Processi
       SO.org:278
       id:5c91241d-3da3-47e6-b27a-9afe7e0b4ff0
       :university:
       Componente del OS: =CPU Scheduler= - Sceglie processi in coda di ready - si attiva ogni 50/100 secondi - crea…

0.860 (+1.7σ)  Sistemi Operativi > Gestione Processi > Scheduling > Implementazione > Scheduler
       SO.org:628
       id:5c91241d-3da3-47e6-b27a-9afe7e0b4ff0
       :university:
       anche Short Term Scheduler decide quale processo in coda di ready sara' eseguito quando: 1. il processo in esecuzione passa…

0.854 (+1.5σ)  Microkernel Based Systems > Kernel Level > Scheduling > in Microkernel Based Systems
       microkernel_based_systems.org:194
       id:ad8e431b-7af6-4eb9-99a7-41af9cd0c4ce
       :erasmus:university:compsci:
       Different ideas: - Brian Ford - CPU Inheritance Scheduling + event \to mk \to root scheduler \to particular scheduler +…

0.850 (+1.4σ)  Microkernel Based Systems > Kernel Level > IPC
       microkernel_based_systems.org:29
       id:ad8e431b-7af6-4eb9-99a7-41af9cd0c4ce
       :erasmus:university:compsci:
       To send messages between threads you don't save and restore those register. The receiving end will declare beforehand to the…

[model load 733ms · query embed 8ms · search over 3038 vectors 1.0ms]

The top note's title — Sistemi Operativi — shares no word with the question. Finding what you can describe but cannot name is the whole point of org-semantic, and each score carries a σ because a raw cosine cannot be read without one.

One note in that vault is UTF-16, and index says so once per index rather than passing over it in silence.

Most of that three-quarters of a second is the model loading, paid once per process. For anything interactive, run org-semantic serve instead: it keeps the model and the vectors resident, and answers in 7–9 ms by meaning or 3 ms by word — fast enough to search as you type.

What it touches

Your notes are read and never written: everything it builds goes in one .org-semantic/ directory beside them, and deleting it leaves the vault exactly as it was. Nothing about them leaves the machine — no service, no account, no API key, no telemetry. The only thing that ever touches the network is fetching the model and a small language classifier, once each, after which it works offline. The manual has the detail, and lists more public vaults if you would rather not start with your own notes.

Installing the binary

Both halves need it: the Emacs package drives this binary rather than carrying one of its own.

Prebuilt ones are on the releases page for Apple Silicon macOS, Linux (x86_64 and arm64) and Windows, the macOS build Developer ID signed and notarized. Unpack it into org-semantic/ under your user-emacs-directory and Emacs finds it with nothing configured at all — that is org-semantic-install-directory, which is searched before exec-path. For shell use, anywhere on your PATH does.

Or build it, which needs a Rust toolchain and nothing else — no Python, no system ONNX Runtime, no package manager:

cargo install --git https://github.com/alberti42/org-semantic

Either way the embedding model downloads on first use.

Documentation

The site is generated from docs/manual.org, which is the canonical documentation; make html builds it locally into public/.

About

Semantic and lexical search for a large collection of org-mode notes. One static binary, no database, no Python.

Resources

Stars

3 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages