Skip to content

pkg/cli: fix flaky TestZipIncludeAndExcludeFilesDataDriven test#162526

Merged
trunk-io[bot] merged 1 commit intocockroachdb:masterfrom
Abhinav1299:fix-flaky-zip-test
Feb 6, 2026
Merged

pkg/cli: fix flaky TestZipIncludeAndExcludeFilesDataDriven test#162526
trunk-io[bot] merged 1 commit intocockroachdb:masterfrom
Abhinav1299:fix-flaky-zip-test

Conversation

@Abhinav1299
Copy link
Contributor

The test was failing because the debug zip output now includes
non-deterministic CPU profile files that weren't being filtered.
Stored CPU profiles in the cpuprof/ directory have timestamped names
like cpuprof.2026-01-27T08_53_56.910.67.pprof, and error files like
cpu.pprof.err.txt are created when live CPU profiling fails during
test execution.

The debug zip has two mechanisms for collecting CPU profiles. Live
CPU profiles are collected via collectCPUProfiles and stored as
cpu.pprof. Stored CPU profiles are collected via collectFileList
with FileType_CPU from the cpuprof/ directory.

This fix adds regex patterns to trimNonDeterministicZipOutputFiles
to filter out these files, similar to how other non-deterministic
outputs like memprof, goroutine_dump, and job-related files are
already handled.

Fixes: #161849
Part of: CRDB-59115
Epic: none
Release note: none

The test was failing because the debug zip output now includes
non-deterministic CPU profile files that weren't being filtered.
Stored CPU profiles in the cpuprof/ directory have timestamped names
like cpuprof.2026-01-27T08_53_56.910.67.pprof, and error files like
cpu.pprof.err.txt are created when live CPU profiling fails during
test execution.

The debug zip has two mechanisms for collecting CPU profiles. Live
CPU profiles are collected via collectCPUProfiles and stored as
cpu.pprof. Stored CPU profiles are collected via collectFileList
with FileType_CPU from the cpuprof/ directory.

This fix adds regex patterns to trimNonDeterministicZipOutputFiles
to filter out these files, similar to how other non-deterministic
outputs like memprof, goroutine_dump, and job-related files are
already handled.

Fixes: cockroachdb#161849
Part of: CRDB-59115
Epic: none
Release note: none
@Abhinav1299 Abhinav1299 requested review from a team as code owners February 5, 2026 08:50
@Abhinav1299 Abhinav1299 requested review from aa-joshi, dhartunian and visheshbardia and removed request for a team February 5, 2026 08:50
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@cockroach-teamcity
Copy link
Member

🔴 Sysbench [SQL, 3node, oltp_read_write]
Metric Old Commit New Commit Delta Note
🔴 sec/op 9.801m ±1% 9.919m ±1% +1.20% p=0.003 n=15
allocs/op 7.963k ±0% 7.978k ±0% ~ p=0.134 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/9d5f581/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/9d5f581d483443be0bc62b67c3f729655f9440f5/bin/pkg_sql_tests benchdiff/9d5f581/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/9d5f581/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/75d90d9/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/75d90d9d6be36e83f23a3916b3f353ec892aa944/bin/pkg_sql_tests benchdiff/75d90d9/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/75d90d9/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

benchdiff --run=^BenchmarkSysbench/SQL/3node/oltp_read_write$ --old=75d90d9 --new=9d5f581 ./pkg/sql/tests
⚪ Sysbench [KV, 3node, oltp_read_only]
Metric Old Commit New Commit Delta Note
sec/op 2.957m ±0% 2.962m ±0% ~ p=0.967 n=15
allocs/op 2.103k ±0% 2.103k ±0% ~ p=0.397 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/9d5f581/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/9d5f581d483443be0bc62b67c3f729655f9440f5/bin/pkg_sql_tests benchdiff/9d5f581/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/9d5f581/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/75d90d9/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/75d90d9d6be36e83f23a3916b3f353ec892aa944/bin/pkg_sql_tests benchdiff/75d90d9/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/75d90d9/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_read_only$ --old=75d90d9 --new=9d5f581 ./pkg/sql/tests
⚪ Sysbench [KV, 3node, oltp_write_only]
Metric Old Commit New Commit Delta Note
sec/op 2.764m ±2% 2.741m ±2% ~ p=0.967 n=15
allocs/op 4.152k ±0% 4.151k ±0% ~ p=0.492 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/9d5f581/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/9d5f581d483443be0bc62b67c3f729655f9440f5/bin/pkg_sql_tests benchdiff/9d5f581/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/9d5f581/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/75d90d9/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/75d90d9d6be36e83f23a3916b3f353ec892aa944/bin/pkg_sql_tests benchdiff/75d90d9/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/75d90d9/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_write_only$ --old=75d90d9 --new=9d5f581 ./pkg/sql/tests
Artifacts

download:

mkdir -p new
gcloud storage cp gs://cockroach-microbench-ci/artifacts/9d5f581d483443be0bc62b67c3f729655f9440f5/21704924747-1/\* new/
mkdir -p old
gcloud storage cp gs://cockroach-microbench-ci/artifacts/75d90d9d6be36e83f23a3916b3f353ec892aa944/21704924747-1/\* old/

built with commit: 9d5f581d483443be0bc62b67c3f729655f9440f5

@cockroach-teamcity cockroach-teamcity added the X-perf-check Microbenchmarks CI: Added to a PR if a performance regression is detected and should be checked label Feb 5, 2026
Copy link
Contributor

@aa-joshi aa-joshi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Please verify the CI unit test failure.

@aa-joshi reviewed 1 file and all commit messages, and made 1 comment.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @dhartunian and @visheshbardia).

@Abhinav1299 Abhinav1299 removed the X-perf-check Microbenchmarks CI: Added to a PR if a performance regression is detected and should be checked label Feb 6, 2026
@Abhinav1299
Copy link
Contributor Author

CI test failure was intermittent. Its fixed now.

@Abhinav1299
Copy link
Contributor Author

TFTR!
bors r+

@blathers-crl
Copy link

blathers-crl bot commented Feb 6, 2026

Bors is deprecated. Please use /trunk merge instead.

@Abhinav1299
Copy link
Contributor Author

/trunk merge

@trunk-io
Copy link
Contributor

trunk-io bot commented Feb 6, 2026

😎 Merged successfully - details.

@trunk-io trunk-io bot merged commit f2dd849 into cockroachdb:master Feb 6, 2026
34 of 35 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cli: TestZipIncludeAndExcludeFilesDataDriven failed

3 participants