The following are four rudimentary to do apps each tested with HitchStory, a StrictYAML based FOSS python integration testing framework.
Unlike with other testing frameworks these declarative stories can:
- autogenerate beautiful documentation with a minimal amount of effort.
- rewrite themselves from program output
- are type-safe
To do App | Storytests | Cool Features | Story Engine |
---|---|---|---|
Website | add todo, correct spelling | Autogenerated add todo docs and correct my spelling docs including playwright generated GIFs and screenshots. | engine.py |
REST API | add todo, correct spelling | Autogenerated docs, API responses in story self-rewrite | engine.py |
Interactive command line app | add todo, correct spelling | Autogenerated docs, Command line outputs in story self-rewrite | engine.py |
Python API | add todo, correct spelling | Autogenerated docs, print() outputs from python in stories self-rewrites | engine.py |
All of the above tests are also running on Github Actions with no tweaks:
Podman must be installed on your system first.
All other functionality is automated and can be run via one of the four key.sh scripts.
To begin:
$ git clone https://github.com/hitchdev/examples.git
$ cd examples/commandlib -OR- restapi -OR- website -OR- pythonapi
$ ./key.sh make
$ ./key.sh regression
This command can be used to craft a new feature and do acceptance test driven development on it:
$ ./key.sh atdd correct
"correct" is a unique keyword used in the name of one of the stories.
If you change the REST API JSON response, the command line output, etc. and run "./key.sh build" and then run this it will rewrite that story:
$ ./key.sh ratdd correct
This will regenerate all of the markdown docs for that project:
$ ./key.sh docgen
For the website project, the screenshots and video artefacts used by the markdown can be regenerated by running:
$ ./key.sh recordings
Warning You might have to fork this project and push changes to preview the github-flavored markdown with videos and images.
This will delete everything created to run these tests (container/volume):
$ ./key.sh clean all
Note It must be run in each project folder you ran ./key.sh make in.
The four folders contain four versions of the same project - this great little to do app built by Oleg Vinokurov which was built with a command line, REST and web interface.
For the pythonapi project everything is just run in the Hitch container (test and code-under-test in separate virtualenvs).
The interactive command line app and REST API have a podman-in-podman:
graph TD;
HitchContainer-->AppContainer;
While the website app is tested with a parent hitch container building and running the app and playwright containers:
graph TD;
HitchContainer-->AppContainer;
HitchContainer-->PlaywrightContainer;
This is done to segregate the test code from the application code.
- Integrate containerized postgres running with all of the apps, seeded with given preconditions.
- Mock the passage of time with a step - implement reminders into the to do apps.
- Handle a REST API response which returns a different UUID each time it is called.
- Demonstrate story inheritance (e.g. logging in story -> add todo) using all of the currently documented inheritance features listed here.
Have suggestions? What kind of gnarly integration tests would you find most interesting? Raise an issue if you have more ideas.