Skip to content

Add Zotero as a literature search and retrieval database - #157

Draft
hechth with Copilot wants to merge 3 commits into
mainfrom
copilot/add-local-zotero-query-database
Draft

hechth with Copilot wants to merge 3 commits into
mainfrom
copilot/add-local-zotero-query-database

Conversation

Copilot AI commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Adds a read-only Zotero backend, built on pyzotero, so a user's own Zotero library (zotero.org or the local desktop app) can be searched for literature and used for paper retrieval. It implements the same interfaces as EuropePMC.

from pathlib import Path
from aoptk.literature.databases.zotero import Zotero
from aoptk.literature.query import Query

# remote
z = Zotero(Path("storage"), Path("figures"), Query("cadmium"),
           library_id="123456", api_key="...", collection="ABCD1234")
# local desktop app (Zotero running, local API enabled)
z = Zotero(Path("storage"), Path("figures"), Query("cadmium"), local=True)

pubs = z.get_publications(z.get_ids())
  • New backend (literature/databases/zotero.py)
    • Implements GetID, GetMetadata, GetAbstract, GetPDF and GetPublication.
    • Constructor parameters: library_id, library_type, api_key, collection, local and full_text_search.
    • local=True defaults library_id to 0 and rejects group libraries. A remote client without library_id raises ValueError.
    • Searches top-level items only (attachments and notes are excluded), scoped to the collection if one is given. The item list is fetched once per instance and cached.
    • IDs are DOI, then PMID, then PMCID (parsed from extra), then the Zotero item key.
    • Full text comes from Zotero's full-text index. If that is unavailable (e.g. on the local API), it falls back to extracting text from the PDF attachment with PyMuPDF.
    • Image attachments are converted to PNG and returned as figures. tables and figure_descriptions are empty because Zotero has no JATS XML.
    • Files are stored as storage/<id>.{txt,pdf}. / and other unsafe characters in IDs (DOIs) are replaced with _.
    • Only Zotero read calls are used; the library is never modified.
  • Optional dependency
    • New extra aoptk[zotero] (pyzotero>=1.15.2), imported lazily.
    • The module imports without pyzotero, and constructing Zotero without it raises an actionable ImportError.
    • uv.lock is regenerated, and the CI test job now syncs with --extra zotero.
  • Unsupported Query filters
    • full_text_subset, only_preprint, exclude_preprint, licensing and date raise NotImplementedError.
    • date raises because the tuple's semantics (exact date vs. range) are unclear. This is a candidate for a client-side filter if wanted.
  • Tests and docs
    • tests/test_zotero_mocked.py uses a fake client and needs no network or pyzotero.
    • New zotero_local pytest marker, reserved for future local-API tests.
    • README section on usage and limitations.

Not included: gated integration tests against a real or local Zotero, and an example script. The code has only been exercised against the fake client.

Co-authored-by: hechth <12066490+hechth@users.noreply.github.com>
Copilot AI changed the title [WIP] Add local Zotero as literature query and paper retrieval database Add Zotero as a literature search and retrieval database Oct 12, 2026
Copilot AI requested a review from hechth October 12, 2026 16:19
Co-authored-by: hechth <12066490+hechth@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add local Zotero as literature query and paper retrieval database

2 participants