Skip to content
JordanCoinPublic

About

Local-first FOIA automation with AI-powered document analysis. Your data never leaves your machine. πŸ”’πŸ‡ΊπŸ‡Έ

Topics

Resources

Stars

11 stars

Watchers

0 watching

Forks

Repository files navigation

OpenFOIA

Local-first investigation toolkit for journalists, researchers, and citizens.

Your data never leaves your machine. Works offline. Works everywhere.

One real government document β†’ 45 entities, 166 relationships, extracted locally in seconds. No cloud. No API keys.

Entity relationship graph β€” 45 entities, 166 relationships extracted from a Palantir police surveillance contract pulled from DocumentCloud

Built-in document reader with entity highlighting β€” click any entity to jump to it in the source text, with a link to the original document


Install

With Claude Code (recommended)

If you use Claude Code, install the OpenFOIA plugin β€” you get the CLI plus a built-in copilot that knows every command:

/plugin marketplace add JordanCoin/openfoia
/plugin install openfoia@openfoia
/foia-install

The plugin ships a skill that teaches Claude how to run FOIA investigations, plus slash commands for the core loop: /foia-search, /foia-investigate, /foia-graph, /foia-install. The last one bootstraps the CLI on your machine.

Shell install (no Claude Code needed)

curl -fsSL https://raw.githubusercontent.com/JordanCoin/openfoia/main/install.sh | bash

Works on Linux, macOS, and Windows (WSL).

New to OpenFOIA? Run openfoia guide after installing.

What It Does

Find the records. Read them page by page. Record findings with citations. Draft the request to the body that actually holds the record, under the law that body answers to, and track its clock. All locally, all offline, all purgeable.

openfoia init                                          # 53 federal agencies pre-loaded
openfoia case new "Gifted program, Detroit 1990s"      # one question, one case
openfoia jurisdiction show us-mi                       # the law a Michigan school district answers under
openfoia agency add --name "Detroit Public Schools Community District" -j us-mi -l school_district --email foia.request@detroitk12.org
openfoia records search "Palantir" --source documentcloud --json --rank police,contract   # search 10M+ public documents, ranked
openfoia records fetch --from hits.json                # pull every hit's text, page by page, into the case
openfoia records fetch CIA-RDP96-00789R002200230001-2 --source cia   # CIA reading room PDF, OCR/ingest locally
openfoia docs grep '"school board" OR WISC' -C 120     # every passage, with page numbers, offline
openfoia case note "No document names minors" --doc 3f2a --page 51 --quote "..." --tier says
openfoia analyze graph --name investigation --no-text --view   # graph scoped to the case
openfoia case export -f md -o report.md                # findings with citations, sources, query log
openfoia request mark REQ-2026-001 denied              # starts the appeal clock the statute sets

Features

PDF text extraction Compiled 2.6MB binary, zero dependencies. 99.5% accuracy across 480K chars. Reports each page as text layer, garbled or image only, and only the last two get OCR
Entity extraction 4-tier: LLM β†’ GLiNER β†’ spaCy β†’ Regex. 100% recall with a 2GB local model
Relationship graphs Interactive graph with built-in document reader. Click entities to read source text with highlights
Cross-reference MuckRock, DocumentCloud, OpenCorporates, SEC EDGAR, OpenSanctions, ICIJ Offshore Leaks
Claude Code skill The plugin teaches Claude the same loop with the same commands, so an agent and a human produce the same cited result
Cases and citations One case per question: searches, documents, findings (says / infer / speculation) with document and page, exported as a cited report
Any US jurisdiction Federal FOIA plus all 50 states and DC: statute, clock, fees, residency and appeal path per law, sourced from the RCFP Open Government Guide, with a verified flag and sources on every file. Add a school district, city, county or university as a public body under its state law
Request clocks Response, extension, tolling, appeal and still-interested deadlines computed from the law that applies, never from a federal default. deadlines check tells you what is overdue
File requests Email, fax (Twilio), physical mail (Lob) to 53 federal agencies or any body you add
Data interchange Import/export FollowTheMoney format (Aleph, OpenAleph, OpenSanctions)
Encrypted storage SQLCipher AES-256. Decoy profile mode
Forensic purge 3-pass overwrite, shell history scrub, free space fill
Portable mode openfoia portable β€” everything stays on the USB, nothing on the host
Metadata stripping Strips EXIF, PDF author, DOCX revision history on file and web-UI ingest. Email attachments and archived web pages are stored as received β€” see THREAT_MODEL.md

PDF Extraction Engine

A compiled 2.6MB binary extracts text directly from PDF structures β€” no OCR needed for born-digital documents. Uses a multi-level resolution cascade to handle broken fonts, missing encodings, and government form PDFs that other tools can't read.

99.5% accuracy across 480K characters in 50 test PDFs. 100% on every FOIA document tested. Zero runtime dependencies. Falls back to OCR (Tesseract) for scanned documents.

OpenFOIA calls the extractor with the large-document runtime profile by default (--profile large). To control concurrency per process, set:

  • OPENFOIA_PDF_EXTRACT_MAX_WORKERS (for example 4 or 8)
  • Optional profile override: OPENFOIA_PDF_EXTRACT_PROFILE=standard|large

Reading and searching stored documents

Everything below runs against the local database, with no network access.

openfoia docs list                                  # stored documents, pages, source
openfoia docs show 3f2a --page 12                   # one page of text (omit --page for all)
openfoia docs grep Hocking                          # every passage that mentions Hocking
openfoia docs grep '"school board" OR WISC' -C 80   # FTS5 phrases, OR, NEAR(a b, 10)
openfoia docs grep audit --doc 3f2a --json          # one document, machine-readable

Results cite real page numbers for DocumentCloud documents and ingested PDFs. Documents stored before page support, or whose page text could not be fetched, are kept as a single page (docs show flags this when the source page count is known). If your SQLite build lacks FTS5 (some SQLCipher builds), docs grep does a plain substring search instead and prints a notice.

Entity extraction picks the best backend it finds. On a local model the LLM validation step can take minutes per document. To skip it, force a local backend with openfoia analyze extract <id> --tier gliner (or spacy, regex), or set OPENFOIA_AI_PROVIDER=none, which wins over any API key or Ollama URL in the config file or environment.

Which law applies

Federal FOIA reaches only federal executive agencies. A school district, city, county, police department or public university answers under its state's public-records act, with that state's clock, fees and appeal path. OpenFOIA carries one record per law, built from the RCFP Open Government Guide, each with its sources and a verified flag. Unknown values stay unknown; nothing is invented.

openfoia jurisdiction list                        # 51 verified, 1 unverified
openfoia jurisdiction show us-tx                  # statute, clock, fees, residency, appeal path, sources
openfoia agency add --name "Austin ISD" -j us-tx -l school_district --custodian-title "Public Information Officer"
openfoia request status REQ-2026-001              # every clock with days left and the rule behind it
openfoia request mark REQ-2026-001 extended       # pushes the due date by the statute's extension
openfoia deadlines check                          # overdue responses, appeals, still-interested replies

Which rung of the system holds your record, and how to ask at each, is in The Records Ladder.

Data Sources

openfoia records search supports these public-records sources. Each search sends its query to the selected third party without a confirmation prompt, so treat queries as sensitive investigation data and run one only when you accept that disclosure.

Source What Access
MuckRock Completed FOIA requests with downloadable response documents API token (OPENFOIA_MUCKROCK_TOKEN)
DocumentCloud Public documents with pre-extracted text Public API
CIA FOIA Reading Room Declassified CIA records (CREST, MORI); records fetch <CREST or MORI> --source cia downloads and ingests the PDF Public site; a bot filter can block scripted access, reported as "blocked"
National Archives Catalog Archival descriptions and OCR text for accessioned federal records; records fetch <NAID> --source nara Free key by email (OPENFOIA_NARA_API_KEY), 10,000 queries a month, counted locally
OpenCorporates Global company registrations, ownership, directors, and filings Free basic search
SEC EDGAR US public-company filings No API key
USAspending Federal contracts, grants, loans, and other awards No API key
ProPublica Nonprofit Explorer IRS Form 990 filings and US nonprofit data No API key
GovInfo Congressional, court, regulatory, and other US government documents Free API key; demo key available
OpenFEC Federal campaign-finance contributions and committees Free API key; demo key available
Regulations.gov Federal rulemaking documents and public comments Free API key; demo key available
OpenSanctions Sanctions lists and politically exposed persons Cross-reference source; free non-commercial access
ICIJ Offshore Leaks Panama, Pandora, and Paradise Papers Cross-reference source; local CSV download

Documentation

Guide What
Journalist Guide Full walkthrough β€” start here if you're new
The Records Ladder Which rung of the US public-records system holds your record, from the school district to NARA, and how to ask
Tails OS Running from Tails for maximum privacy
USB Install Encrypted USB portable setup
Air-Gapped Offline deployment overview

Privacy

Everything runs on 127.0.0.1. Token-authenticated. Encrypted at rest (optional). Forensically purgeable. Works offline. Open source.

For sensitive investigations: openfoia serve --tor

Portable mode: openfoia portable β€” data stays on the USB, nothing touches the host.

When you're done: openfoia purge --secure --yes

Uninstall

curl -fsSL https://raw.githubusercontent.com/JordanCoin/openfoia/main/uninstall.sh | bash

Legal Context for Contributors

Writing this code is legal. The US DOJ stated in August 2025: "Merely writing code, without ill intent, is not a crime." Open source encryption is exempt from US export controls. The Tor Project, Signal, SecureDrop, and Tails set the precedent. More context.

Credits

Built by people who believe in freedom of information. Inspired by MuckRock, DocumentCloud, and the Reporters Committee for Freedom of the Press.

License

AGPL-3.0 β€” Keep it open.


"Democracy dies in darkness. FOIA is how we turn on the lights."

About

Local-first FOIA automation with AI-powered document analysis. Your data never leaves your machine. πŸ”’πŸ‡ΊπŸ‡Έ

Topics

Resources

Stars

11 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages