Repository navigation
refactor(document): consolidate five forked PDF extractors onto one canonical path #13886
Description
Five independent PDF text extractors exist; three are live and two are fully orphaned (728 lines, zero callers).
| Location | Lib | Wired |
|---|---|---|
| autobot-backend/api/knowledge.py:1065 | pypdf | live - KB upload |
| autobot-backend/media/document/pipeline.py:104 | pypdf | live - via media/manager.py:40 |
| autobot-backend/knowledge/connectors/content_extraction.py:39 | PyPDF2 | live - gdrive.py:40, onedrive.py:44 |
| autobot-backend/utils/document_parser.py:124 | pypdf | orphaned - 0 callers |
| autobot-backend/utils/document_extractors.py:126 | pypdf | orphaned - 0 callers |
SECURITY-ADJACENT: the connector fork runs PyPDF2 (unmaintained) while requirements.txt:23 pins pypdf>=6.15.0 with the comment SECURITY UPDATE (improved permission checks). Google Drive and OneDrive ingestion runs on the library we upgraded away from.
Verified orphan status: grep -rn document_parser|document_extractors --include=*.py outside those two files returns nothing, not even tests.
Per the no-deletion rule the two orphans are WIRED IN, not removed: document_parser.py carries format breadth (DOC/DOCX/XLSX/PPT/PPTX/ODT/ODS/ODP/ODG) the live paths lack, and document_extractors.py carries directory-batch logic. Fold both into the survivor.
Survivor should be media/document/pipeline.py - it already returns structured ProcessingResult with confidence, page_count, document_info and an unavailable-vs-error distinction.
Blocks #13884 (empty-layer detection) and the OCR fallback from shipping five times.
Activity
mrveiss commentedon Aug 31, 2026
Duplicate of #13893, which shipped this exact consolidation and is merged (
0b913d1f59, PR #13924).Verification against current
Dev_new_guitree (2026-08-31):autobot-backend/media/document/extraction.pyis the single canonical extractor (extract_pdf,extract_docx,extract_document).api/knowledge.py:1095—_extract_pdf_documentcallsmedia.document.extraction.extract_pdfmedia/document/pipeline.py— importsextract_pdf/extract_docxfrommedia.document.extractionknowledge/connectors/content_extraction.py:16— importsextract_pdf/extract_docxfrommedia.document.extractionutils/document_parser.py:15,124,142—_parse_pdf/_parse_docxdelegate tomedia.document.extractionutils/document_extractors.py:40,140,182— delegates tomedia.document.extractiongrep -rn "PyPDF2" --include=*.py .→ only historical comments/docstrings, no live import.grep -n "PyPDF2" autobot-backend/requirements.txt→ only a comment noting its removal (refactor(document): consolidate five forked PDF extractors onto one canonical path #13893), no dependency line.test_only_the_canonical_module_constructs_a_pdf_readerandtest_pypdf2_is_not_imported_anywhere(autobot-backend/tests/test_document_extraction_canonical_13893.py) pin exactly this as a structural guard against the fork regrowing.document_parser.pystill carries the DOC/DOCX/XLSX/PPT/PPTX/ODT/ODS/ODP/ODG format breadth (called fromknowledge/documents.py),document_extractors.py's directory-batch logic is called fromknowledge/documents.py::DocumentExtractor.Gate 2:
pipeline-scripts/check-issue-close-refs.sh 13886→ clear, no forward-tracking references.Closing as a duplicate; #13893 carries the acceptance criteria and evidence.