Disable prune_by_limit when output ordering is required

[zhuqi-lucas]

Summary

Fix incorrect query results when ORDER BY + LIMIT is combined with partially-matched row groups.

prune_by_limit discards partially-matched row groups when enough fully-matched groups can satisfy the limit. This is only safe for unordered scans — with output ordering, a partially-matched group may contain rows that sort before any fully-matched group.

Root cause

ParquetOpener had no awareness of output ordering. It called prune_by_limit unconditionally whenever a limit was set.

Fix

Add preserve_order: bool to ParquetOpener:

• Set to true when output_ordering is non-empty (in ParquetSource::create_opener)
• Guard prune_by_limit: if let (Some(limit), false) = (limit, preserve_order)

Example

Parquet sorted by [a ASC], 3 rows per RG:

• RG0: a=[1,2,3] — partially matched for a > 2
• RG1: a=[4,5,6] — fully matched

Query: WHERE a > 2 ORDER BY a ASC LIMIT 1

	Result	Correct?
Before (no guard)	a=4 from RG1	❌ skipped RG0
After (preserve_order=true)	a=3 from RG0	✅

Test

test_preserve_order_prevents_limit_pruning — creates a real parquet with 2 RGs, verifies:

• preserve_order=false: prune_by_limit skips RG0, returns a=4
• preserve_order=true: RG0 is kept, returns a=3

[Copilot]

Pull request overview

This PR fixes incorrect results for Parquet scans where ORDER BY + LIMIT is planned as an order-preserving scan and row-group pruning uses prune_by_limit, which can incorrectly drop partially-matched row groups that may contain earlier-sorting rows.

Changes:

• Add a preserve_order flag to ParquetOpener and set it based on FileScanConfig.output_ordering in ParquetSource::create_file_opener.
• Guard prune_by_limit so it is only applied when preserve_order is false.
• Add a regression test ensuring preserve_order=true prevents limit-based pruning from skipping partially-matched row groups.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
datafusion/datasource-parquet/src/source.rs	Propagates ordering requirements from FileScanConfig into the Parquet file opener via preserve_order.
datafusion/datasource-parquet/src/opener.rs	Disables limit pruning when output ordering must be preserved; adds a regression test covering the correctness bug.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.