Use `Display` formatting of `DataType`:s in error messages #17565

emilk · 2025-09-15T09:33:12Z

Related: Improve Display for DataType and Field arrow-rs#8290

Rationale for this change

I believe that error messages should be as readable as possible. Aim for rustc more than gcc.

Display is the nice, user-facing formatter. Debug is for… well, debugging.

What changes are included in this PR?

Change a bunch of {:?} format string to {}. I'm sure I missed a lot of them, because I know of no way to enforce this without

disallowed_methods on trait impls rust-lang/rust-clippy#8581

Are these changes tested?

I assume CI runs cargo test :)

Are there any user-facing changes?

Yes! Error messages should be a bit more readable now, especially once this lands:

Improve Display for DataType and Field arrow-rs#8290

alamb

Thank you @emilk -- this is looking great. Once we get the CI to pass I think we can merge it in

emilk · 2025-09-15T12:13:53Z

Should be green now

[Narrator] it was not

alamb · 2025-09-15T13:45:23Z

I recommend running cargo test locally first to catch all the test errors and fix them in one go, in case you're not already doing so

My personal favorite is cargo nextest run

emilk · 2025-09-15T13:47:35Z

I did, unfortunately cargo test they didn't cover all the test, and cargo test --all-features doesn't compile. Apparently CI does something slightly more complex:

https://github.com/rerun-io/arrow-datafusion/blob/e077920b19867d02b43f836c92bfb61183493ea0/.github/workflows/extended.yml#L109-L119

…and others

emilk · 2025-09-15T14:45:49Z

cargo nextest run passes on my machine now - let's see what CI says

emilk · 2025-09-17T06:04:14Z

Reverted some error messages, and cargo nextest run is still green locally…

emilk · 2025-09-17T11:41:30Z

OMG IS GREEN

timsaucer

Thank you, Emil!

alamb · 2025-09-17T21:52:44Z

gogogogo!

alamb · 2025-09-17T21:52:56Z

Thank you very much @emilk and @timsaucer

This is part of an attempt to improve the error reporting of `arrow-rs`, `datafusion`, and any other 3rd party crates. I believe that error messages should be as readable as possible. Aim for `rustc` more than `gcc`. Here's an example of how this PR improves some existing error messages: Before: > Casting from Map(Field { name: "entries", data_type: Struct([Field { name: "key", data_type: Utf8, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }, Field { name: "value", data_type: Interval(DayTime), nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }]), nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }, false) to Map(Field { name: "entries", data_type: Struct([Field { name: "key", data_type: Utf8, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }, Field { name: "value", data_type: Duration(Second), nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }]), nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }, true) not supported After: > Casting from Map(Field { "entries": Struct(key Utf8, value nullable Interval(DayTime)) }, false) to Map(Field { "entries": Struct(key Utf8, value Duration(Second)) }, true) not supported # Which issue does this PR close? - Closes #7048 - Continues and closes #7051 - Continues #7469 - More improvements coming in #8291 - Sibling PR: apache/datafusion#17565 - Part of #8351 # Rationale for this change DataType:s are often shown in error messages. Making these error messages readable is _very important_. # What changes are included in this PR? ## ~Unify `Debug` and `Display`~ ~The `Display` and `Debug` of `DataType` are now the SAME.~ ~Why? Both are frequently used in error messages (both in arrow, and `datafusion`), and both benefit from being readable yet reversible.~ Reverted based on PR feedback. I will try to improve the `Debug` formatting in a future PR, with clever use of https://doc.rust-lang.org/std/fmt/struct.Formatter.html#method.debug_struct ## Improve `Display` of lists Improve the `Display` formatting of * `DataType::List` * `DataType::LargeList` * `DataType::FixedSizeList` **Before**: `List(Field { name: \"item\", data_type: Int32, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} })` **After**: `List(nullable Int32)` **Before**: `FixedSizeList(Field { name: \"item\", data_type: Int32, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }, 5)` **After**: `FixedSizeList(5 x Int32)` ### Better formatting of `DataType::Struct` The formatting of `Struct` is now reversible, including nullability and metadata. - Continues #7469 ### ~Improve `Debug` format of `Field`~ ~Best understood with this diff for an existing test:~ <img width="1140" height="499" alt="Screenshot 2025-09-07 at 18 30 44" src="https://github.com/user-attachments/assets/794b4de9-8459-4ee7-82d2-8f5ae248614c" /> **EDIT**: reverted # Are these changes tested? Yes - new tests cover them # Are there any user-facing changes? `Display/to_string` has changed, and so this is a **BREAKING CHANGE**. Care has been taken that the formatting contains all necessary information (i.e. is reversible), though the actual `FromStr` implementation is still not written (it is missing on `main`, and missing in this PR - so no change). ---- Let me know if I went to far… or not far enough 😆 --------- Co-authored-by: irenjj <renj.jiang@gmail.com>

# Rationale for this change Despite us having `Display` implementations for `DataType`, a lot of error messages still use `Debug`. See for instance: * apache/datafusion#17565 * #8290 Therefor I want to make sure the `Debug` formatting of `Field` (and, by extension, `DataType`) is not _utterly horrible_. This PR makes things… slightly better. # What changes are included in this PR? Omits fields of `Field` that have their "default" values. # Are these changes tested? Yes, there are new tests. # Are there any user-facing changes? Though this changes the `Debug` formatting, I would NOT consider this a breaking change, because nobody should rely on consistent `Debug` formatting. See for instance https://doc.rust-lang.org/std/fmt/trait.Debug.html#stability

emilk added 4 commits September 15, 2025 11:10

Use Display formatting for DataTypes where I could find them

e0a7f1b

fix

f9fd2d9

More places

99e188f

Less Debug

b1a06d0

emilk mentioned this pull request Sep 15, 2025

Improve Display for DataType and Field apache/arrow-rs#8290

Merged

emilk marked this pull request as ready for review September 15, 2025 09:36

alamb approved these changes Sep 15, 2025

View reviewed changes

emilk added 4 commits September 15, 2025 13:05

Cargo fmt

c30c7e1

More cleanup

787ddd0

Plural types as Display

97bca0d

Fixes

e6bc7a2

emilk added 2 commits September 15, 2025 14:57

Update some more tests and error messages

3bb23cb

Update test snapshot

2f27791

github-actions bot added the sqllogictest SQL Logic Tests (.slt) label Sep 15, 2025

last (?) fixes

6ef4be0

emilk force-pushed the emilk/use-display-formatting branch from 3812ed5 to 6ef4be0 Compare September 15, 2025 13:16

update another slt

6fd1386

emilk added 4 commits September 15, 2025 15:51

Update instructions on how to run the tests

45bf70c

Ignore pending snapshot files in .gitignore

37908ee

Running all the tests is so slow

77c1c15

just a trailing space

bfe8e60

emilk mentioned this pull request Sep 15, 2025

Use Display for DataTypes, not Debug formatting rerun-io/rerun#11219

Merged

Update another test

4ce61e3

emilk added 6 commits September 16, 2025 10:18

Merge branch 'main' into emilk/use-display-formatting

b991e27

Fix markdown formatting

0ebb1bd

Improve Display for NativeType

9e80295

Merge branch 'main' into emilk/use-display-formatting

2788fd1

Update code related to error reporting of NativeType

1e6bf50

Revert some formatting

fe9b0ec

emilk added 2 commits September 17, 2025 08:39

fixelyfix

9315cea

Another snapshot update

55653f3

emilk changed the title ~~Use Display formatting of DataType:s in error messagwe~~ Use Display formatting of DataType:s in error messages Sep 17, 2025

timsaucer approved these changes Sep 17, 2025

View reviewed changes

alamb added this pull request to the merge queue Sep 17, 2025

Merged via the queue into apache:main with commit 631f9ab Sep 17, 2025
32 checks passed

emilk mentioned this pull request Sep 23, 2025

Use more compact Debug formatting of Field apache/arrow-rs#8424

Merged

emilk mentioned this pull request Sep 26, 2025

New lint: disallowed-trait-usage rust-lang/rust-clippy#15765

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Use `Display` formatting of `DataType`:s in error messages #17565

Use `Display` formatting of `DataType`:s in error messages #17565

Uh oh!

emilk commented Sep 15, 2025 •

edited

Loading

Uh oh!

alamb left a comment

Uh oh!

emilk commented Sep 15, 2025 •

edited

Loading

Uh oh!

alamb commented Sep 15, 2025

Uh oh!

emilk commented Sep 15, 2025 •

edited

Loading

Uh oh!

emilk commented Sep 15, 2025

Uh oh!

emilk commented Sep 17, 2025

Uh oh!

emilk commented Sep 17, 2025

Uh oh!

timsaucer left a comment

Uh oh!

alamb commented Sep 17, 2025

Uh oh!

alamb commented Sep 17, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Use Display formatting of DataType:s in error messages #17565

Use Display formatting of DataType:s in error messages #17565

Uh oh!

Conversation

emilk commented Sep 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

alamb left a comment

Choose a reason for hiding this comment

Uh oh!

emilk commented Sep 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alamb commented Sep 15, 2025

Uh oh!

emilk commented Sep 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

emilk commented Sep 15, 2025

Uh oh!

emilk commented Sep 17, 2025

Uh oh!

emilk commented Sep 17, 2025

Uh oh!

timsaucer left a comment

Choose a reason for hiding this comment

Uh oh!

alamb commented Sep 17, 2025

Uh oh!

alamb commented Sep 17, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Use `Display` formatting of `DataType`:s in error messages #17565

Use `Display` formatting of `DataType`:s in error messages #17565

emilk commented Sep 15, 2025 •

edited

Loading

emilk commented Sep 15, 2025 •

edited

Loading

emilk commented Sep 15, 2025 •

edited

Loading