-
Notifications
You must be signed in to change notification settings - Fork 2.6k
Testing framework for runtimes #3479
Comments
Okay, this is a bit vague: testing the runtime can be hard but in my opinion ONLY because of the boilerplate needed to set up the externalities. Afterwards, you can perfectly simulate anything that you might need including processing blocks with just single calls to basic meta-modules like system and timestamp. What is quite annoying is:
If the issue concerns above then we have another tracking issue for it: paritytech/polkadot-sdk#367 |
I think there are two separate things being spoken of:
For the latter we have |
ahh, I see. Okay so on top of your recommendation, I usually use it for e2e tests over the real substrate node runtime. Though at the moment it is using the binaries of |
@kianenigma |
@kianenigma @tomusdrw I think the bigger issue is less the lack of facilities, and more the lack of documentation on how to use them. Should I open another issue about developer docs? |
The intention is good, but 95% of the Substrate code base lacks documentation on how to use it. I don't think it's useful to open an issue specifically about |
The only thing that's lacking IMO is documentation. I'm in agreement that the externalities setup is a little tedious/mysterious for testing of runtime modules. I can also see some level of property-based testing to be valuable -- for instance, rather than calling runtime functions directly, you call them indirectly where you have to say under what circumstances a function is callable. The specific case I have in mind is whether it's called before initialization or after -- trait implementations for e.g. Testing the whole thing in tandem is going to be difficult simply due to the number of combinations. I think we will probably need a separate tool for creating such tests, like the Ethereum state-test suite. |
@rphmeier hopefully https://github.com/runtimeverification/polkadot-verification will deal with this (by proving the absence of bugs). |
Testing runtimes is currently difficult in practice. While it seems to only involve passing a few blocks to the runtime and seeing what the runtime does, there are no convenience functions for this.
The text was updated successfully, but these errors were encountered: