Support deletion vector merge-on-read

[JingsongLi]

What changed

• support deletion-vectors.merge-on-read=true for deduplicate primary-key batch snapshot reads
• include level-0 files, preserve overlapping key ranges in one split, and apply deletion vectors before key merging
• keep non-key predicates out of pre-merge file pruning and enforce them after merging
• load deletion vectors lazily per physical file and cap DV merge fan-in at 256 sorted runs
• match Java behavior by treating merge-on-read as an ignored option when deletion vectors are disabled
• document the option and add core plus DataFusion regression coverage

Why

Deletion-vector tables normally hide uncompacted level-0 files until compaction. Java Paimon allows batch readers to opt into merge-on-read so recent level-0 data is visible immediately. The Rust reader parsed the option but did not implement the corresponding scan and read path.

Impact

Deduplicate primary-key tables with deletion vectors can now expose uncompacted data in batch snapshot queries while preserving primary-key, deletion-vector, and residual-filter correctness. The default remains unchanged. Streaming and changelog behavior are not affected, and partial-update or aggregation with DV merge-on-read remain unsupported.

Validation

• cargo test -p paimon --lib — 2244 passed, 1 ignored
• cargo test -p paimon-datafusion --test pk_tables — 54 passed
• cargo clippy -p paimon --all-targets -- -D warnings
• cargo clippy -p paimon-datafusion --test pk_tables -- -D warnings
• cargo fmt --all -- --check
• git diff --check

[leaves12138]

Reviewed the deletion-vector merge-on-read implementation in detail. The scan planning, per-file deletion-vector application before key merging, post-merge residual filtering, incremental-read isolation, and bounded merge fan-in all look correct. Targeted core and DataFusion tests pass, formatting is clean, and all CI checks are green. LGTM.