Support csv truncated rows in datafusion

[zhuqi-lucas]

Which issue does this PR close?

Support csv truncated rows in datafusion

• Closes #17464

Rationale for this change

Support csv truncated rows in datafusion

What changes are included in this PR?

Support csv truncated rows in datafusion

Are these changes tested?

Yes

Are there any user-facing changes?

New feature support.

[xudong963]

LGTM

[alamb]

Thanks @zhuqi-lucas !

[zhuqi-lucas]

Thank you @xudong963 , @alamb!

[sm4rtm4art]

Hey,

I'm currently updating the DataFrame API documentation and noticed something about CsvReadOptions.

The method truncated_rows is publicly available in the source, but it doesn't seem to be rendered on the docs.rs page here:
https://docs.rs/datafusion/latest/datafusion/datasource/file_format/options/struct.CsvReadOptions.html

although it is implemented here :

https://github.com/apache/datafusion/blob/main/datafusion/core/src/datasource/file_format/options.rs#L94-L98

I just wanted to confirm if this is expected before I document it.

It seems like a useful builder method to mention.

Thanks!

[zhuqi-lucas]

Hey,

I'm currently updating the DataFrame API documentation and noticed something about CsvReadOptions.

The method with_truncated_rows is publicly available in the source, but it doesn't seem to be rendered on the docs.rs page here: https://docs.rs/datafusion/latest/datafusion/datasource/file_format/options/struct.CsvReadOptions.html

although it is implemented here :

https://github.com/apache/datafusion/blob/main/datafusion/core/src/datasource/file_format/options.rs#L94-L98

I just wanted to confirm if this is expected before I document it.

It seems like a useful builder method to mention.

Thanks!

It works well @sm4rtm4art , thank you!

[sm4rtm4art]

Thanks for looking into this!

I might be missing something, but I've re-checked the docs.rs page and the with_truncated_rows method still doesn't seem to be appearing.

https://docs.rs/datafusion/latest/datafusion/datasource/file_format/options/struct.CsvReadOptions.html

Is it possible there's a delay in the documentation build, or am I looking at a cached version?

@xudong963

Thanks again for your help!

[alamb]

We released 51 just this morning. truncated rows now seems to be available:

• Release DataFusion 51.0.0 (Nov 2025) #17558

https://docs.rs/datafusion/latest/datafusion/common/arrow/csv/struct.ReaderBuilder.html#method.with_truncated_rows