feat: add compact_tables to ConvertDocumentsOptions - #4032
Conversation
Exposes the compact_tables option, already supported by export_to_markdown()/save_as_markdown(), for avoiding column padding in markdown table export. Defaults to False, no behavior change. Signed-off-by: Alexey Rybalchenko <alexryba@gmail.com>
|
✅ DCO Check Passed Thanks @rbx, all your commits are properly signed off. 🎉 |
Merge Protections🟢 Merge protection satisfied — ready to merge. Show 1 satisfied protection🟢 Enforce conventional commitMake sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
|
This adds the field, but nothing consumes it yet: Measured on docling-serve v1.29.0 (docling-slim 2.117.0, docling-core 2.88.0). Same xlsx, 137 267 B, 5 000 rows x 10 columns, one cell of 20 000 characters:
A production file from a downstream indexing service: 1 638 201 B xlsx -> 317 567 892 characters, 4 915 705 with the flag; padding was ~86% of the characters in its table chunks, and the conversion was OOM-killing their workers. The wiring on the jobkit side is mechanical and has a precedent in the very same functions: docling-jobkit#108 did this before I can open the docling-jobkit PR - either after this one lands, or now marked as depending on it. Which do you prefer? |
Adds
compact_tables: bool = FalsetoConvertDocumentsOptions, exposing thecompact_tablesparameter thatDoclingDocument.export_to_markdown()andsave_as_markdown()already support - added to fix unbounded whitespace growth in markdown table export (#2907). A single wide cell in a column (e.g. a merged banner/disclaimer row) currently forces every other cell in that column to pad out to match, which can turn a small, sparse table into megabytes of near-entirely-whitespace output.This was originally submitted directly to
docling-jobkitin docling-jobkit#108, which added the field toConvertDocumentsOptionsthere. That PR was redirected here becauseConvertDocumentsOptionshas since moved to live in this repo (docling-jobkit's copy is now just a compatibility re-export). This PR is that redirected change.I've also looked at where this options could be used in docling-jobkit and there seem to be quite a few places. I prepared a potential PR for that, but not very confident if it covers everything. Can open if you like.