-
Notifications
You must be signed in to change notification settings - Fork 121
Fix autoharness
termination test & print metadata in alphabetical order
#3971
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix autoharness
termination test & print metadata in alphabetical order
#3971
Conversation
In some previous PR I had already run into the issue of timeouts occurring before the unwinding limit was reached, at which point I increases the default timeout to 60 seconds instead of 30 seconds. Can we instead adjust the timeout for just this test (via command-line options) to 5 minutes? |
I thought about doing this, but I thought that was a long time to let a single test run. I suppose it's okay to make one exception though. |
9f8ef96
to
7de0c1b
Compare
I've split the termination test into two tests, one to test the unwind bound and one for the timeout. The unwind test sets the timeout to 5m, but at least testing locally it never gets close to that amount, so I think we're good. |
2cfe4dc
Bump Kani version to 0.61.0. Github-generated release notes: ## What's Changed * Fix CHANGELOG of 0.60.0 by @qinheping in #3925 * Bump tests/perf/s2n-quic from `d88faa4` to `8670e83` by @dependabot in #3928 * Update toolchain to 2025-03-04 by @qinheping in #3927 * Install the right toolchain for HEAD and BASE checks in `verify-std-check.yml` by @remi-delmas-3000 in #3920 * Automatic cargo update to 2025-03-10 by @github-actions in #3926 * Automatic toolchain upgrade to nightly-2025-03-05 by @github-actions in #3929 * Upgrade toolchain to nightly-2025-03-07 by @tautschnig in #3931 * Upgrade toolchain to nightly-2025-03-12 by @tautschnig in #3933 * Automatic toolchain upgrade to nightly-2025-03-13 by @github-actions in #3934 * Update CBMC dependency to 6.5.0 by @tautschnig in #3936 * Automatic toolchain upgrade to nightly-2025-03-14 by @github-actions in #3937 * Automatic toolchain upgrade to nightly-2025-03-15 by @github-actions in #3938 * Automatic toolchain upgrade to nightly-2025-03-16 by @github-actions in #3939 * Automatic toolchain upgrade to nightly-2025-03-17 by @github-actions in #3940 * Automatic cargo update to 2025-03-17 by @github-actions in #3941 * Autoharness: Don't panic on `_` argument and add `_autoharness` suffix to GOTO files by @carolynzech in #3942 * Implement `f16` and `f128` cases in `codegen_float_type` by @carolynzech in #3943 * Support function implementations of known built-ins by @tautschnig in #3945 * Autoharness: metadata improvements and enable standard library application by @carolynzech in #3948 * Autoharness: `--list` option by @carolynzech in #3952 * Add support for anonymous nested statics by @carolynzech in #3953 * Automatic cargo update to 2025-03-24 by @github-actions in #3954 * Bump tests/perf/s2n-quic from `8670e83` to `324cf31` by @dependabot in #3955 * Document behavior of checked_size_of_raw and is_inbounds by @rajath-mk in #3956 * Upgrade toolchain to 2025-03-18 by @zhassan-aws in #3959 * Remove unstable-features from code formatting script by @zhassan-aws in #3962 * Remove CI job to update features/verify-rust-std by @tautschnig in #3963 * Make is_inbounds public by @rajath-mk in #3958 * Enable Kani to work with a stable toolchain by @zhassan-aws in #3964 * Automatic cargo update to 2025-03-31 by @github-actions in #3966 * Add support for struct field accessing in loop contracts by @thanhnguyen-aws in #3970 * Bump tests/perf/s2n-quic from `324cf31` to `d0aff82` by @dependabot in #3968 * Clarify `is_inbounds` docs by @carolynzech in #3974 * Upgrade toolchain to 2025-04-01 by @carolynzech in #3973 * Remove remaining `--enable-unstable` mentions by @carolynzech in #3978 * Clean up unused dependencies by @zhassan-aws in #3981 * Automatic toolchain upgrade to nightly-2025-04-02 by @github-actions in #3983 * Update dependencies per `cargo-outdated` by @carolynzech in #3982 * Fix `autoharness` termination test & print metadata in alphabetical order by @carolynzech in #3971 * Fix cargo invocations to only use `pkg_args` where appropriate by @carolynzech in #3984 **Full Changelog**: kani-0.60.0...kani-0.61.0 By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.
#3968 failed in the merge queue because the
cargo_autoharness_termination
test assumed that the recursion test wouldn't time out, but it did. This doesn't happen often, since this test has worked up until now, but in the interest of having robust tests, this PR splits the termination test into two:autoharness_timeout
, which tests that the default timeout of 60s kicks inautoharness_unwind
, which tests that the default unwind bound of 20 kicks inThe unwind test specifies a timeout of 5m to ensure that the unwind bound takes effect before the timeout.
While I was fixing this, I also fixed some inconsistencies with the printed metadata:
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.