Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to use quickcheck with E2E testing #1647

Open
cmichi opened this issue Feb 9, 2023 · 1 comment
Open

Add ability to use quickcheck with E2E testing #1647

cmichi opened this issue Feb 9, 2023 · 1 comment
Assignees
Labels
A-ink_e2e [ink_e2e] Work item B-research Research task that has open questions that need to be resolved.

Comments

@cmichi
Copy link
Collaborator

cmichi commented Feb 9, 2023

We should allow for the possibility of using quickcheck with our E2E testing framework.

The problem here is that the #[quickcheck] macro already generates code which contains the #[test] attribute (whereas we need #[ink_e2e::test]. But I think we can access the quickcheck functionality via some other way directly calling functions on the crate.

ToDo 1

Please investigate if we can find a way to make this work:

#[quickcheck]
#[ink_e2e::test]

I think having this ☝️ would be the best API, but I think it's unlikely that we can get this to work. The other way would be sth like

#[ink_e2e::test(quickcheck = …)]

and then pass the arguments through. So quickcheck would be a direct dependency of ink_e2e and we would likely call a Rust function directly on that crate.

ToDo 2: Benchmark

After #1642 I'm not sure how running quickcheck on top of our E2E testing will behave. We should have some numbers of how long it takes to e.g. fuzz a contract message 10_000 times.

@cmichi cmichi added B-research Research task that has open questions that need to be resolved. A-ink_e2e [ink_e2e] Work item labels Feb 9, 2023
@SkymanOne
Copy link
Contributor

SkymanOne commented Feb 10, 2023

@cmichi
After some digging around: we can use quickcheck(<func_name> as <func_signature>) directly in our E2E tests as shown here. It behaves exactly as macros. However, that would require "diving" into the test function block and parsing build_message calls

e.g.

let insert = ink_e2e::build_message::<MappingsRef>(contract_id)
    .call(|contract| contract.insert_balance(1_000));
let size = client
    .call(&ink_e2e::alice(), insert, 0, None)
    .await
    .expect("Calling `insert_balance` failed")
    .return_value();

and turning them into sub-functions which can be accepted as args by quickcheck. That is actually where #1644 would come handy as it would simplify parsing because contract's associated functions will have a simple mapping of primitive types (e.g. usize -> usize), and we can plug them directly into quickcheck function

Otherwise, I personally can't see another way of doing it or using quickcheck! macro or #[quickcheck] attr without changing the API of E2E

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-ink_e2e [ink_e2e] Work item B-research Research task that has open questions that need to be resolved.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants