Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add unit tests for concrete playback, refactor util functions for easier testing #2188

Merged
merged 15 commits into from
Feb 9, 2023

Conversation

jaisnan
Copy link
Contributor

@jaisnan jaisnan commented Feb 6, 2023

Description of changes:

This PR adds

  1. Unit tests related to concrete playback outside of the existing regression tests.
  2. Removes randomized layout and metadata as a parameter to generate the unit tests
  3. Changes hash to use the pretty_name instead of the mangled_name.

Resolved issues:

Partly resolves #1502

Call-outs:

  1. Thanks to @sanjit-bhat for the work in PR refactor(kani-driver, concrete_playback): process unit tests line by line, modularize functions, and add unit tests #1625
  2. It changes the string that is to be hashed and added as a hash to the function. The naming schema hasn't yet changed to the user by this PR but will be in the coming few PR's.
  3. The refactoring (removing dependency on metadata and randomized_layout) allows the functions to be tested and in the future, will allow the unit tests to be generated line by line.

Testing:

  • How is this change tested? Unit and regression tests

  • Is this a refactor change? Yes

Checklist

  • Each commit message has a non-empty body, explaining why the change was made
  • Methods or procedures are documented
  • Regression or unit tests are included, or existing tests cover the modified code
  • My PR is restricted to a single feature or bugfix

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.

@jaisnan jaisnan requested a review from a team as a code owner February 6, 2023 16:50
@jaisnan jaisnan changed the title Add unit tests for concrete playback w/ util functions Add unit tests for concrete playback Feb 6, 2023
Copy link
Contributor

@adpaco-aws adpaco-aws left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've read the code but I'm still confused about the change in the function names of the tests. Are the generated unit tests runnable using these names without any modifications?

@jaisnan
Copy link
Contributor Author

jaisnan commented Feb 6, 2023

I've read the code but I'm still confused about the change in the function names of the tests. Are the generated unit tests runnable using these names without any modifications?

No, those generated unit tests are not runnable. I can make another revision fixing the names as well (will just need one more layer of formatting the names of the functions).

@celinval
Copy link
Contributor

celinval commented Feb 7, 2023

I've read the code but I'm still confused about the change in the function names of the tests. Are the generated unit tests runnable using these names without any modifications?

No, those generated unit tests are not runnable. I can make another revision fixing the names as well (will just need one more layer of formatting the names of the functions).

I love the idea of adding the name of the original harness. However, I think this new name schema will break the --inplace UX though, wouldn't it?

@jaisnan
Copy link
Contributor Author

jaisnan commented Feb 7, 2023

I've read the code but I'm still confused about the change in the function names of the tests. Are the generated unit tests runnable using these names without any modifications?

No, those generated unit tests are not runnable. I can make another revision fixing the names as well (will just need one more layer of formatting the names of the functions).

I love the idea of adding the name of the original harness. However, I think this new name schema will break the --inplace UX though, wouldn't it?

Yes, I am trying to find a way to make the UX work, but if not, I'll just go back to the old naming scheme.

@celinval
Copy link
Contributor

celinval commented Feb 8, 2023

Can you remove all the modules and only keep the function name?

@jaisnan jaisnan requested a review from adpaco-aws February 8, 2023 18:27
Copy link
Contributor

@celinval celinval left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's awesome. Thanks @jaisnan

Once you merge this, can you please update our documentation as well to reflect the new convention please?

kani-driver/src/args.rs Outdated Show resolved Hide resolved
kani-driver/src/concrete_playback.rs Show resolved Hide resolved
kani-driver/src/concrete_playback.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@zhassan-aws zhassan-aws left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the PR changes the name of concrete playback tests, the PR title should mention that.

kani-driver/src/concrete_playback.rs Outdated Show resolved Hide resolved
@jaisnan jaisnan changed the title Add unit tests for concrete playback Change generated unit test names, Add unit tests for concrete playback Feb 8, 2023
Copy link
Contributor

@celinval celinval left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, can you please either extend an existing test or add a new test for the new name schema? Thanks

Copy link
Contributor

@zhassan-aws zhassan-aws left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit confused about the description in the PR:

Functions weren't given their scope prefixes before which lead to multiple generated unit tests in the same file being called "kani_concrete_playback_harness", which is confusing to the user. This change renames them to use their scope as prefixes if available (using the "pretty name"). For example - "kani_concrete_playback_second::harness".

Can you clarify with an example?

kani-driver/src/args.rs Show resolved Hide resolved
kani-driver/src/concrete_playback.rs Outdated Show resolved Hide resolved
kani-driver/src/concrete_playback.rs Outdated Show resolved Hide resolved
kani-driver/src/concrete_playback.rs Show resolved Hide resolved
tests/ui/concrete-playback/use-function-name/main.rs Outdated Show resolved Hide resolved
@jaisnan jaisnan changed the title Change generated unit test names, Add unit tests for concrete playback Add unit tests for concrete playback, refactor util functions for easier testing Feb 9, 2023
@jaisnan jaisnan merged commit 647892b into model-checking:main Feb 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add unit tests for kani-driver concrete playback components
4 participants