Skip to content

Comments

[CI] Don't fail-fast#1396

Merged
KCarretto merged 1 commit intomainfrom
ci-no-fail-fast
Dec 24, 2025
Merged

[CI] Don't fail-fast#1396
KCarretto merged 1 commit intomainfrom
ci-no-fail-fast

Conversation

@KCarretto
Copy link
Collaborator

Updates our tests CI to not fail fast, allowing all tests to finish (without cancelling tests on other operating systems), improving our flaky detection.

@KCarretto KCarretto requested a review from hulto December 24, 2025 18:12
@github-actions
Copy link
Contributor

Summary

Tests 📝 Passed ✅ Failed ❌ Skipped ⏭️ Other ❓ Flaky 🍂 Duration ⏱️
1751    ±0 1750    ±0 1    ±0 0    ±0 0    ±0 0    ±0 1ms    ±0

Previous Results

Build 🏗️ Result 🧪 Tests 📝 Passed ✅ Failed ❌ Skipped ⏭️ Other ❓ Flaky 🍂 Duration ⏱️
#3236 1751 1750 1 0 0 0 26.5s

Insights

Average Tests per Run Total Flaky Tests Total Failed Slowest Test (p95)
1751 0 1 43.6s

Fail Rate

Fail Rate 0.06%    ±0
Test 📝 Results 📊 Passed ✅ Failed ❌ Fail Rate (%) 📈
eldritch: assets::copy_impl::tests::test_embedded_copy 3 2 1 33.33    ↑33.33
eldritch: assets::copy_impl::tests::test_embedded_copy 3 2 1 33.33    ↑33.33
eldritch: assets::copy_impl::tests::test_embedded_copy 3 2 1 33.33    ↑33.33

Failed Tests

Failed Tests
❌ eldritch: assets::copy_impl::tests::test_embedded_copy
thread 'assets::copy_impl::tests::test_embedded_copy' (12483) panicked at lib/eldritch/src/assets/copy_impl.rs:227:9

Trace:

thread 'assets::copy_impl::tests::test_embedded_copy' (12483) panicked at lib/eldritch/src/assets/copy_impl.rs:227:9:
assertion failed: contents.contains("hello from an embedded shell script")
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Slowest Tests

Test 📝 Results 📊 Duration (avg) ⏱️ Duration (p95) ⏱️
eldritch: random::string_impl::tests::test_string_uniform 2 41.8s 43.6s
eldritch: random::string_impl::tests::test_string_uniform 2 41.8s 43.6s
eldritch: process::info_impl::tests::test_info_default 2 4.2s 8.2s
eldritch: process::info_impl::tests::test_info_default 2 4.2s 8.2s
eldritch: pivot::port_scan_impl::tests::test_portscan_return_type_starlark_dict_from_interpreter 2 3.8s 7.5s
eldritch: pivot::port_scan_impl::tests::test_portscan_return_type_starlark_dict_from_interpreter 2 3.8s 7.5s
eldritch: pivot::ssh_copy_impl::tests::test_pivot_ssh_copy 2 4.8s 7.5s
eldritch: pivot::ssh_copy_impl::tests::test_pivot_ssh_copy 2 4.8s 7.5s
eldritch: pivot::ssh_exec_impl::tests::test_pivot_ssh_exec 2 4.8s 7.5s
eldritch: pivot::ssh_exec_impl::tests::test_pivot_ssh_exec 2 4.8s 7.5s

🍂 No flaky tests in this run.

Github Test Reporter by CTRF 💚

cd vscode/
npm install -g @vscode/vsce
vsce package
code --install-extension eldritch-0.0.3.vsix
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👀chat is this real???

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shhhh 🙊

@KCarretto KCarretto merged commit 698d91c into main Dec 24, 2025
7 of 8 checks passed
@KCarretto KCarretto deleted the ci-no-fail-fast branch December 24, 2025 18:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants