Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crossprop tests for bitwise and blockinfo #309

Merged
merged 9 commits into from
Feb 9, 2024
Merged

Crossprop tests for bitwise and blockinfo #309

merged 9 commits into from
Feb 9, 2024

Conversation

csgui
Copy link
Collaborator

@csgui csgui commented Jan 25, 2024

Initial pull request with some crossprop tests:

  • bitwise
  • blockinfo

I would like to validate the approach in this PR to make it the base for other implementations.

@csgui csgui requested review from obycode and Acaccia January 25, 2024 13:50
clar2wasm/tests/wasm-generation/bitwise.rs Outdated Show resolved Hide resolved
clar2wasm/tests/wasm-generation/blockinfo.rs Outdated Show resolved Hide resolved
clar2wasm/tests/wasm-generation/blockinfo.rs Outdated Show resolved Hide resolved
Copy link

codecov bot commented Jan 25, 2024

Codecov Report

Attention: 8 lines in your changes are missing coverage. Please review.

Comparison is base (c0d3f3c) 91.38% compared to head (97b971e) 91.45%.
Report is 8 commits behind head on main.

Files Patch % Lines
clar2wasm/src/tools.rs 55.55% 4 Missing and 4 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #309      +/-   ##
==========================================
+ Coverage   91.38%   91.45%   +0.06%     
==========================================
  Files          40       40              
  Lines       13803    13936     +133     
  Branches    13803    13936     +133     
==========================================
+ Hits        12614    12745     +131     
+ Misses        558      545      -13     
- Partials      631      646      +15     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@csgui csgui requested review from Acaccia and obycode January 29, 2024 16:35
clar2wasm/tests/wasm-generation/blockinfo.rs Outdated Show resolved Hide resolved
clar2wasm/tests/wasm-generation/blockinfo.rs Outdated Show resolved Hide resolved
clar2wasm/tests/wasm-generation/blockinfo.rs Outdated Show resolved Hide resolved
Copy link
Collaborator

@Acaccia Acaccia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really have anything else to add. The other issue with crosscheck with env is already noted.

clar2wasm/tests/wasm-generation/main.rs Outdated Show resolved Hide resolved
@csgui csgui requested review from Acaccia and obycode January 30, 2024 18:58
@csgui
Copy link
Collaborator Author

csgui commented Jan 31, 2024

Tests are failing randomly when checking the expression (get-block-info? time uXX). The difference in the results are always by 1. Seems a false positive.

Any take on that @Acaccia @obycode ?

Test failed: assertion `left == right` failed: Compiled and interpreted results diverge! (get-block-info? time u5)
compiled: Ok(Some(Optional(OptionalData { data: Some(UInt(1706712683)) })))
interpreted: Ok(Some(Optional(OptionalData { data: Some(UInt(1706712684)) })))

@Acaccia
Copy link
Collaborator

Acaccia commented Feb 2, 2024

@csgui I don't know why this test gives you this result, but it would be nice to get the explanation before merging this PR. Because for now, I have no idea if it's the test that is wrong, or if there is a bug in the code.

@csgui
Copy link
Collaborator Author

csgui commented Feb 2, 2024

@Acaccia Yep, that is the reason I have opened this discussion. Maybe more pair of 👀 can bring some light to this issue.

@obycode
Copy link
Collaborator

obycode commented Feb 2, 2024

In the simulation, when the datastore is created, the time is saved, then each block is assigned a time that is this genesis time plus height * some constant. This means that if the creation times of the two datastores (one for the interpreter and one for clarity-wasm) are different, then this value will be different, in this case by 1 second.

@Acaccia
Copy link
Collaborator

Acaccia commented Feb 2, 2024

two datastores (one for the interpreter and one for clarity-wasm) are different

That seems like a serious bug to me. Not for the code concerned by this PR but for all the crosscheck tests in general. I'm not sure we can rely on the compared results if both have different initial values for the datastore.

Can't we create only one at the beginning and clone it?

@csgui
Copy link
Collaborator Author

csgui commented Feb 2, 2024

🤔 ... fair point @Acaccia . We are relying on different TestEnvironment instances for both interpreted and compiled results.

Seems that we should tackle that.

@csgui csgui force-pushed the tests/crossprop branch 2 times, most recently from 17f3f26 to 1a4a4ed Compare February 6, 2024 11:33
Copy link
Collaborator

@Acaccia Acaccia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is supposed to be the introduction to crossprop, would it be possible to add documentation to each crosscheck functions?
That would be helpful for newcomers.

@csgui csgui requested a review from Acaccia February 6, 2024 18:06
clar2wasm/src/tools.rs Outdated Show resolved Hide resolved
clar2wasm/tests/wasm-generation/blockinfo.rs Outdated Show resolved Hide resolved
@csgui csgui requested a review from Acaccia February 7, 2024 10:53
Acaccia
Acaccia previously approved these changes Feb 7, 2024
Copy link
Collaborator

@Acaccia Acaccia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@csgui csgui requested review from obycode and Acaccia February 8, 2024 14:24
Copy link
Collaborator

@obycode obycode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @csgui!

@csgui csgui enabled auto-merge February 9, 2024 09:39
@csgui csgui added this pull request to the merge queue Feb 9, 2024
Merged via the queue into main with commit 9ec68a6 Feb 9, 2024
6 of 7 checks passed
@csgui csgui deleted the tests/crossprop branch February 9, 2024 09:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants