feat(examples): add setup tooling so the examples can actually be run - #591
feat(examples): add setup tooling so the examples can actually be run#591williedoran-neo4j wants to merge 15 commits into
Conversation
Fourteen examples call load_dotenv(), but python-dotenv was never declared. It resolves today only transitively, through another extra, so the examples work by accident and would break if that path changed. Sits on the lock regeneration in the previous commit, which keeps the lock diff here to the two lines this actually adds. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The examples read five environment variables between them and there was no template, no mention of them in examples/README.md, and no committed guidance on where to put a key. examples/.env.example is that template. Every credential line is commented out on purpose: an empty FOO= would blank out an already exported variable when the file is sourced, so a line is uncommented only when it carries a real value. .gitignore already covered .env; this widens it to .env.* with an explicit !.env.example negation, so .env.local and similar variants cannot be committed by accident while the template stays tracked. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Running the examples locally needed Neo4j with APOC, and the vector store examples needed Weaviate, Qdrant or Pinecone. That knowledge was spread across the README's test section, three per-store READMEs and tests/e2e/docker-compose.yml, which exists for the e2e suite rather than for the examples. Profiles keep the default cheap: bare 'up' starts Neo4j alone, and the vector stores or a containerised Ollama are opt-in. Two things this adds over the e2e stack: - Healthchecks, so 'up --wait' blocks until the services actually answer. The e2e stack has none and CI polls readiness by hand, which leaves local users racing connection-refused. - pinecone-local, an in-memory emulator that ignores API keys, so the Pinecone examples no longer require a hosted account. It deliberately binds the same ports as tests/e2e/docker-compose.yml, so the two stacks are mutually exclusive; the file header says so. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Identical to the file in PR neo4j#578 - reviewed there, reproduced here so this branch stands alone off main. Nothing to re-review in this commit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Answers one question per example: what does it need in order to run. Used by the setup tooling in the following commits, and by check_examples.py once the commit after this one rewires it. Requirements come from three places, in decreasing order of how well they can be inferred: - Packages, from an AST import scan. Examples import library symbols rather than provider SDKs - OpenAILLM, not openai - so the scan maps neo4j_graphrag symbols to extras, not just top-level modules. - Env vars and datastores, read out of os.getenv calls and connection URIs appearing as string literals. - Everything else - APOC, a pre-existing index, outbound internet - leaves no reliable trace in the source, so it is declared per path. It also detects the three examples that import a vector from examples/data as a bare module, which only resolves with that directory on sys.path. Stdlib only, so it runs before uv sync. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
providers_used() matched provider names as substrings of the whole source, so any file containing the word 'google' counted as a Vertex example and OpenAIEmbeddings could not be told from AzureOpenAIEmbeddings. It now delegates to the shared requirement model, which resolves providers from imported symbols. --live gains two things from that model: - It skips snippets, which have nothing to run, instead of executing them and reporting a pass. - It checks whether the services an example needs are actually up. A stopped Neo4j previously surfaced as FAIL on ~30 examples, which reads as 'the examples are broken' when the truth is 'nothing was running'. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Foundations for the setup tooling in the following commits, kept apart because they carry the rules that matter most: - envfile writes credentials to .env only, at mode 0600, and masks every value it echoes back. It also overlays .env onto the process, so a credential that exists only in that file is still visible to SDKs that read the real environment - without it a correctly configured AWS_PROFILE reports as 'no credentials'. - probes answer 'is this present' for packages, commands, a Neo4j with APOC and its indexes, and pulled Ollama models. Everything degrades to a negative answer rather than raising, so the doctor still runs before uv sync. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
One entry per service the examples talk to: which extra it needs, which env vars, what its free tier gives you, and where to get a key. Validators call the provider's list-models endpoint. That is free, and it proves the key is live rather than merely well-formed, which matters because the installer refuses to store a key that does not validate - a bad value in .env turns a clear 'unset' into a confusing 401. Bedrock and Vertex carry a credential probe instead of an env var: their credentials come from a CLI login or a named profile, so testing for a variable would report a working setup as broken. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The read-only half of the setup tooling. Resolves every example against the shared requirement model, probes what is present, and reports the single first thing standing in the way of each one, grouped so the biggest wins surface first. It writes nothing, reaches no further than a TCP connect, and exits 0 unless --strict, so it is safe to run casually. Credentials are reported as set or unset and never printed, hashed, or partially revealed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Walks the same ground as the doctor, in tiers, so the free and local providers work before any cloud account is involved: extras and a local Neo4j with the indexes examples assume, then API keys, then local runtimes, then cloud. Every step is skippable and safe to re-run. Keys are read with getpass, validated against the provider, and written to .env only on success. Tier 3 lives in its own module because it has to ask who is running it. Reading an Aura dev environment through omni is useless to an outside contributor, and telling a Neo4j developer to run 'aws configure' is the wrong advice; neither can be inferred safely, so it prompts. Cloud account identifiers are read at run time and never persisted, since they change whenever an environment is rebuilt. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
python scripts/setup_examples.py --check # what is missing
python scripts/setup_examples.py # walk through fixing it
Argument parsing and tier dispatch only; the work lives in the
examples_setup package.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ollama_llm.py and ollama_embeddings.py shipped a literal <model_name>
placeholder, so they could not run without being edited first.
ollama_tool_calls.py instead hardcoded mistral:latest, which 404s
unless you happen to have pulled that model.
There is no model name that is right for everyone - it depends on what
you have pulled - so it is now an argument, with a default that has
been run:
ollama pull llama3.2
python examples/customize/llms/ollama_llm.py llama3.2
Defaults are llama3.2 for the two LLM examples and nomic-embed-text for
embeddings. All three pass with no arguments once those are pulled.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
examples/README.md indexes what the examples demonstrate but says nothing about how to run one: no API keys, no extras, no Docker, no mention that sixteen of them talk to a remote demo database. SETUP.md covers that, including a per-example table of provider, extras, env vars and services, and which providers are free without a credit card - every non-OpenAI provider turns out to have a free or local path. It also records what running all of them turned up, so the next person does not debug their own setup for hours over a defect that is already known: three library bugs that stop Cohere and MistralAI working at all, four dead model references, and the three examples that need examples/data on PYTHONPATH. Each carries the replacement that was verified working. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
scripts/ is not a package and is not installed, so tests under tests/ cannot import check_examples. Add it to pythonpath under the same top-level module names mypy already resolves it under; importing it as scripts.check_examples instead would give mypy the same file under two module names. Coverage is unaffected: [tool.coverage.run] source is ["src"], so nothing under scripts/ can enter the measured set. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…detects Review follow-ups on the setup tooling: - The Gemini validator carried the key in the query string, and an unreachable host interpolated the whole URL into a string the installer prints. Any offline, DNS or proxy failure put the key in the user's scrollback. Sent as a header now, and only the host reaches the message. - .env was created by write_text() and chmod-ed afterwards, so a fresh file held the key world-readable until the next statement. Created at 0600 through a temp file and os.replace, so an interrupted write also cannot truncate the credentials already there. - The Ollama tier pulled qwen3:8b while the examples default to llama3.2, so a completed setup run left them failing, and it closed by telling the user to edit a <model_name> placeholder that argparse replaced earlier in this branch. - Removed the placeholder machinery entirely: no example contains it any more, so the doctor blocker and the --live skip it fed were unreachable. - Three config-file examples name their LLM in YAML/JSON, invisible to the import scan, and were reported as needing nothing. SERVICE_RULES entries can now declare extras, providers and env vars for cases like this. - install_hints() could only ever return [], because analyse() narrowed modules to exactly the set install_hints() filters back out. It now surfaces the real case: tools_retriever_example.py imports requests, which nothing declared. Declared it, and python-dotenv, in the examples extra. Also: linked SETUP.md from examples/README.md, gave t2v-transformers a healthcheck (python3, since the image has no wget or curl) and documented why pinecone-local cannot have one, honoured --non-interactive in the Aura tier, and stopped the doctor parsing all 104 examples four times and re-resolving cloud credentials once per example. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
bd3de7d to
569f8e2
Compare
|
Closing this in favour of five smaller PRs. At 15 commits and ~3,900 lines it was too much to ask of a reviewer, and going back over it turned up a fair amount that should not have shipped at all — a hand-maintained 86-row table with rows that were already wrong, a It also carried a Neo4j-internal Replaced by:
Net effect: roughly 1,400 fewer lines than this PR, in pieces that can be reviewed and merged independently. |
Linear: GENKGB-1857
Follow-up to #578, which fixed the gpt-5 breakage in the OpenAI examples. That work exposed a second problem: there is no way to run the other examples without reverse-engineering their requirements from the source.
examples/README.mdis a pure link index — no mention of API keys, extras, Docker, or the fact that sixteen examples talk to a remote demo database. What setup knowledge exists is spread across the README's Tests section, three per-store READMEs, andtests/e2e/docker-compose.yml, which exists for the e2e suite rather than the examples. CI uses a fourth, divergent path: hand-rolleddocker run, no compose.What's here
scripts/setup_examples.py— a doctor and an interactive installer.--checkresolves every example against a shared requirement model, probes packages, env vars, databases and containers, and reports the single first thing blocking each one, grouped so the biggest wins surface first. It writes nothing and exits 0 unless--strict. On a machine with keys set but no services running it reports 52 runnable, 34 blocked, 18 snippets — and 33 of those 34 are blocked only on a stopped Neo4j or Ollama, which is the point: bring the stack up and they run.The installer works in tiers, so the free and local providers work before any cloud account is involved: extras and local Neo4j, then API keys, then local runtimes, then cloud. Every step is skippable and safe to re-run. Tier 3 asks whether you're using a Neo4j Aura dev environment or your own cloud accounts — the internal path is useless to an outside contributor, and the reverse is wrong for a Neo4j developer, and neither can be inferred safely.
scripts/example_requirements.py— one model of what each example needs, shared withcheck_examples.py. Package requirements come from an AST import scan mapping library symbols to extras, because an example importsOpenAILLMrather thanopenai, so the dependency is invisible from its imports alone. Env vars and datastores are read fromos.getenvcalls and connection URIs. Anything that leaves no trace in the source — APOC, a pre-existing index, outbound internet, or a component named only in a YAML/JSON config — is declared per path.This also replaces
providers_used()incheck_examples.py, which matched provider names as substrings of the whole file: any file containing the word "google" counted as a Vertex example, andOpenAIEmbeddingscouldn't be told fromAzureOpenAIEmbeddings.--livenow also skips snippets rather than running them and reporting a pass, and checks whether the services an example needs are actually up — a stopped Neo4j previously surfaced as FAIL on ~30 examples, which reads as "the examples are broken" when the truth is "nothing was running".examples/docker-compose.yml— Neo4j with APOC by default; avectordbprofile adding Weaviate, Qdrant and Pinecone Local; an optionalollamaprofile. Services declare healthchecks soup --waitblocks until they actually answer; the e2e stack has none and CI polls readiness by hand, which leaves local users racing connection-refused. The one exception ispinecone-local, whose image ships no shell and no binary a healthcheck could run — documented in the file header. Pinecone Local is an in-memory emulator that ignores API keys, so that path no longer needs a hosted account. Same ports as the e2e stack, so the two are mutually exclusive — the file header says so.examples/.env.exampleandexamples/SETUP.md— a credential template, and per-example docs covering provider, extras, env vars and services, plus which providers are free without a card. Every non-OpenAI provider turns out to have a free or local path.examples/README.mdnow links it.python-dotenvandrequestsdeclared in theexamplesextra — fourteen examples callload_dotenv()andtools_retriever_example.pycalls a web API withrequests, but neither was declared, so both resolved only by accident.Credentials
Keys are read with
getpass, validated against the provider with a free list-models call, and written only to a gitignored.env. A key that doesn't validate is never written — a bad value there turns a clear "unset" into a confusing 401. Nothing is printed in full, and cloud account identifiers are resolved at run time rather than persisted.Two details worth calling out, because both are the kind of thing that is easy to get subtly wrong:
x-goog-api-keyheader, and no error message interpolates a URL, since a query string is logged by every proxy in the path and error strings are printed to the terminal..envis created at mode 0600 rather than created and then chmod-ed, so the secret never exists on disk world-readable, not even briefly. It is written through a temp file andos.replace, so an interrupted write cannot truncate the credentials already there.Tests
tests/unit/scripts/— 44 cases over the requirement model, the.envwriter and the provider validators. No network (urlopen is replaced and a fixture fails the test if anything reaches out), no keys, no services.They are there because they catch real things. Each of these was confirmed to fail against the code as it was before: the key-in-the-URL disclosure, the
.envcreation mode, and the three config-file examples being reported as needing nothing at all. Worth noting for anyone reviewing the permissions test: asserting the final mode of.envproves nothing, since the old and new code both end at 0600 — the test forbids the chmod fixup instead, so the mode has to be right at creation.scripts/is deliberately outside the coverage gate ([tool.coverage.run] source = ["src"]), so these are additive and cannot move the 90% number.For review
The
feat(scripts): add static checks for the examples directorycommit is byte-identical to fix(examples): make OpenAI examples work with gpt-5 #578's copy of that file. It exists so this branch stands alone offmain; it drops out cleanly on rebase once fix(examples): make OpenAI examples work with gpt-5 #578 merges. Nothing to re-review there. The same applies to the one-linechore(tests): put scripts/ on the pytest pathcommit, which both branches need independently.Both were verified rather than assumed — rebasing this branch onto a
maincontaining fix(examples): make OpenAI examples work with gpt-5 #578 reportsdropping … patch contents already upstreamandskipped previously applied commit, 15 commits become 13, with zero conflicts.With both branches together the checker reports 104 files, no problems — verified locally. On this branch alone it reports 22 problems, because this branch carries the checker but not fix(examples): make OpenAI examples work with gpt-5 #578's example fixes.
Commits are ordered to be reviewed one at a time: compose and env template first (no Python), then the requirement model, then the CLI on top, then docs.
Deliberately not fixed, documented in
SETUP.mdinstead: two examples write message history to the read-only demo database and fail withForbidden, and three needPYTHONPATH=examples/data.Type of Change
Complexity
Complexity: Medium
How Has This Been Tested?
Exercised, not just reviewed. Brought the stack up from clean and watched the doctor's runnable count climb as services started. Ran tier 0 and confirmed it creates the four indexes the examples assume — including the
moviePlotsEmbeddingindex on the local database that no script in this repo previously created. Confirmed all five key validators reject an invalid key and write nothing. Stopped Neo4j and confirmed the doctor reports blocked rather than passing. Thet2v-transformershealthcheck was verified by running the image: it has no wget or curl but does ship python3, and the ready endpoint answers 204.Green on
ruff check,ruff format --check,mypy --strict, and the unit suite with coverage at the 90% gate.Checklist
🤖 Generated with Claude Code