PDF text extraction fails for subsetted/custom fonts — shows GLYPH placeholders

[manikrishna-m]

When passing a PDF to DocumentStream using the stream parameter (raw bytes), some PDFs produce garbled text with GLYPH<c=...> placeholders or letters that appear shifted (e.g., 7KH instead of THE). This occurs only for PDFs that use subsetted fonts or custom encodings without a proper ToUnicode map. PDFs created from Word, LaTeX, or InDesign usually work fine, but certain exported PDFs or scanned PDFs fail.

Steps to reproduce:

from docling import DocumentStream

# Using raw PDF bytes
with open("problematic.pdf", "rb") as f:
    doc = f.read()

source = DocumentStream(name="problematic.pdf", stream=doc)
conv_result = self.doc_converter.convert(source)

Observed behavior:

• Output text contains GLYPH<c=...> sequences.
• Letters may appear shifted, as if Caesar-ciphered.

Expected behavior:

• Extracted text should reflect the real content of the PDF.

Environment:

• Docling version: 2.40.0
• Python version: 3.12.3

[dosubot]

PDF text extraction in Docling relies on the pypdfium2 library, which uses font encoding and ToUnicode maps from the PDF to map glyphs to Unicode text. If these mappings are missing or non-standard (as is common with subsetted or custom fonts), pypdfium2 will output placeholder glyphs like GLYPH<c=...> or garbled text, and Docling surfaces this output directly—there’s no built-in detection or remediation for these cases¹.

Passing a file path instead of raw bytes sometimes improves extraction because some PDF libraries can access more metadata or apply different heuristics when working with files, but this is not guaranteed to fix issues with problematic fonts or missing ToUnicode maps².

Docling does support OCR as a fallback, but you need to enable it manually in your pipeline configuration. For example, you can set pipeline_options.ocr_options.force_full_page_ocr = True to force OCR even when a text layer is present³. This is especially useful for scanned PDFs or those with broken font encoding.

There’s currently no automatic detection of font encoding issues or automatic fallback to OCR in Docling, and the difference between stream=bytes and path=file is not documented in detail. Improving detection, fallback, and documentation would be valuable enhancements.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

Footnotes

1. Docling uses pypdfium2 for PDF text extraction, which relies on font encoding and ToUnicode maps ↩

2. InputDocument and DocumentStream distinguish between Path and BytesIO, affecting backend handling ↩

3. How to force OCR for scanned PDFs in Docling ↩

[manikrishna-m]

@dosu I still have that issue even after using the pipeline_options.ocr_options.force_full_page_ocr = True

[dosubot]

If you're still seeing GLYPH placeholders or garbled text after setting pipeline_options.ocr_options.force_full_page_ocr = True, make sure that pipeline_options.do_ocr = True is also set—OCR won't run unless both are enabled, regardless of the force flag (source).

Also, Docling skips images smaller than 5% of the page area for OCR by default, so set bitmap_area_threshold=0 and picture_area_threshold=0 in your pipeline options to guarantee OCR runs on all images (source).

Check that your OCR engine (EasyOCR, Tesseract, RapidOCR, etc.) is installed and working, and that the correct language/model files are available. If you still get empty output, enable debug logging and inspect any warnings or errors—issues like "OSD failed" or "Invalid resolution 0 dpi" often point to missing DPI metadata or misconfigured Tesseract (source).

If possible, share your full pipeline configuration and any error logs for more targeted troubleshooting.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}