Open
Description
I run cargo test from the command line many times every day, always with the default --message-format human value. Usually, I have a quick scan through the list of test cases and look for any red FAILED text.
I have three very minor annoyances about the human message format:
- The list of tests isn't ordered by test name. This makes it hard to see at a glance if the failures are in one module or many.
- The list of tests aren't ordered by pass/fail. This means I have to read through every line to see which tests failed. It also means if you have more tests than terminal rows, you have to scroll to find all the failing tests.
- The green pass/red fail text is on the very end of the line, meaning it never lines up. Because each line is variable width (due to each test name having a variable length), I can't just glance my eyes down a column of text. My eyes have to dart around from column to column.
These are really minor issues, but they slow down my reading comprehension. Given how often I read the cargo test output every day, it might be worth improving its glanceability.
Suggested solution 0
Do nothing. If users want to see failures more easily, they can just read through the more detailed failure output that each failed test outputs. This is a reasonable response. However, given that the summarized list of each test and its pass/fail status exists, I feel it should be as readable as possible.
Suggested solution 1
Add a new format flag, --message-format human-sorted which is identical to --message-format human except for these changes:
- List of tests is ordered by pass/fail status, then by test name. This way, all failing tests are listed towards the end of the output (so users don't have to scroll their terminal), and tests from the same module will be next to each other (so users can tell which modules are failing).
- Pass/fail status appears at the start of the line rather than the end. This way, users can just scan their eyes down a single aligned column of text, instead of needing to move their eyes in a diagonal down-left down-right way.
Here is an example of --message-format human output
running 33 tests
test config::tests::test_cloudflared_args ... ok
test config::tests::test_contains_duplicate ... ok
test monitor::poll::tests::test_event_into_fail ... ok
test monitor::poll::tests::test_event_into_happened ... ok
test monitor::poll::tests::test_event_into_wait ... ok
test monitor::poll::tests::test_event_to_option ... ok
test monitor::poll::tests::test_poll_always_succeed ... ok
test monitor::poll::tests::test_poll_always_wait ... ok
test monitor::poll::tests::test_poll_zero_timeout_wait ... ok
test monitor::poll::tests::test_trimap ... ok
test monitor::report::tests::test_fail_report_sentry_message ... ok
test monitor::report::tests::test_fail_report_sentry_tags ... ok
test monitor::tunnelchecker::tests::test_cname ... ok
test monitor::tunnelchecker::tests::test_count_active_conns ... ok
test monitor::tunnelchecker::tests::test_count_active_conns_malformed ... ok
test monitor::tunnelchecker::tests::test_count_active_conns_no_conns ... ok
test monitor::tunnelchecker::tests::test_count_active_conns_one_unregistered ... ok
test monitor::tunnelchecker::tests::test_is_lb ... ok
test monitor::tunnelchecker::tests::test_parse_tunnel_alphanumeric ... ok
test monitor::tunnelchecker::tests::test_parse_tunnel_ids_many_lines ... ok
test monitor::tunnelchecker::tests::test_parse_tunnel_ids_no_matches ... ok
test monitor::tunnelchecker::tests::test_parse_tunnel_ids_single_line ... ok
test monitor::tunnelchecker::tests::test_parse_tunnel_non_alphanumeric ... ok
test monitor::tunnelchecker::tests::test_parse_tunnel_reg_fail_many_lines ... ok
test monitor::tunnelchecker::tests::test_parse_tunnel_reg_fail_no_matches ... ok
test monitor::tunnelchecker::tests::test_parse_tunnel_reg_fail_with_success_line ... ok
test monitor::tunnelchecker::tests::test_parse_tunnel_reg_success ... ok
test monitor::tunnelchecker::tests::test_register_tunnel_fail_response ... ok
test monitor::tunnelchecker::tests::test_register_tunnel_timeout ... ok
test monitor::tunnelstarter::tests::test_cloudflared_cli_args ... ok
test monitor::tunnelstarter::tests::test_parse_hostname_fail ... ok
test monitor::tunnelstarter::tests::test_parse_hostname_success ... ok
And here is the same test output but with --message-format human-sorted output:
running 33 tests
ok ... test config::tests::test_cloudflared_args
ok ... test config::tests::test_contains_duplicate
ok ... test monitor::poll::tests::test_event_into_fail
ok ... test monitor::poll::tests::test_event_into_happened
ok ... test monitor::poll::tests::test_event_into_wait
ok ... test monitor::poll::tests::test_event_to_option
ok ... test monitor::poll::tests::test_poll_always_wait
ok ... test monitor::poll::tests::test_poll_zero_timeout_wait
ok ... test monitor::poll::tests::test_trimap
ok ... test monitor::report::tests::test_fail_report_sentry_tags
ok ... test monitor::tunnelchecker::tests::test_cname
ok ... test monitor::tunnelchecker::tests::test_count_active_conns
ok ... test monitor::tunnelchecker::tests::test_count_active_conns_malformed
ok ... test monitor::tunnelchecker::tests::test_count_active_conns_no_conns
ok ... test monitor::tunnelchecker::tests::test_count_active_conns_one_unregistered
ok ... test monitor::tunnelchecker::tests::test_is_lb
ok ... test monitor::tunnelchecker::tests::test_parse_tunnel_alphanumeric
ok ... test monitor::tunnelchecker::tests::test_parse_tunnel_ids_many_lines
ok ... test monitor::tunnelchecker::tests::test_parse_tunnel_ids_no_matches
ok ... test monitor::tunnelchecker::tests::test_parse_tunnel_ids_single_line
ok ... test monitor::tunnelchecker::tests::test_parse_tunnel_non_alphanumeric
ok ... test monitor::tunnelchecker::tests::test_parse_tunnel_reg_fail_many_lines
ok ... test monitor::tunnelchecker::tests::test_parse_tunnel_reg_fail_no_matches
ok ... test monitor::tunnelchecker::tests::test_parse_tunnel_reg_fail_with_success_line
ok ... test monitor::tunnelchecker::tests::test_parse_tunnel_reg_success
ok ... test monitor::tunnelchecker::tests::test_register_tunnel_fail_response
ok ... test monitor::tunnelchecker::tests::test_register_tunnel_timeout
ok ... test monitor::tunnelstarter::tests::test_cloudflared_cli_args
ok ... test monitor::tunnelstarter::tests::test_parse_hostname_fail
FAILED ... test monitor::poll::tests::test_poll_always_succeed
FAILED ... test monitor::report::tests::test_fail_report_sentry_message
FAILED ... test monitor::tunnelstarter::tests::test_parse_hostname_success
I personally think this format is much more readable. It requires less eye motion and less scrolling (in the event that the list of tests doesn't fit in your terminal height). I would be happy to implement this change if the community thought it was helpful. I'd suggest keeping human as the default value for --message-format and adding human-sorted as an alternative, to avoid breaking anyone's code that relies on the current default format.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Metadata
Assignees
Labels
Area: `#[test]` / the `test` libraryArea: `#[test]` / the `test` libraryCategory: A feature request, i.e: not implemented / a PR.Category: A feature request, i.e: not implemented / a PR.This change is large or controversial enough that it should have an RFC accepted before doing it.This change is large or controversial enough that it should have an RFC accepted before doing it.
Type
Projects
Status
No status