Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvement of regression tests #969

Open
aronroland opened this issue Apr 1, 2023 · 0 comments
Open

Improvement of regression tests #969

aronroland opened this issue Apr 1, 2023 · 0 comments
Labels
enhancement New feature or request

Comments

@aronroland
Copy link
Collaborator

aronroland commented Apr 1, 2023

Is your feature request related to a problem? Please describe.
The Regression tests at this point only compare new results to old results and it is checked whether the results are bit 4 bit reproducible. However, this is not enough. The regression tests need to be extended in terms of:

  • Performance, e.g. if a pull request results in performance penalties this needs to be explained or rejected.
  • Accuracy, e.g. laboratory experiments or field cases needs to be elaborated in terms of changes in the accuracy. Pull requests resulting in penalties in accuracy needs to be addressed or rejected.
  • Parallel scalability needs to be tested as well, e.g. if the scalability is degraded it needs to be addressed or rejected.
  • Memory usage, if the memory usage is increased, it needs to be addressed or rejected as well.

Describe the solution you'd like
Add performance, accuracy, scalability and memory usage to the metrics.

Describe alternatives you've considered
no alternative

Additional context
none

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant