Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for order agnostic asserts in tremor-cli based test harness #2481

Open
darach opened this issue Apr 12, 2024 · 1 comment
Open
Assignees
Labels
enhancement New feature or request

Comments

@darach
Copy link
Member

darach commented Apr 12, 2024

Describe the problem you are trying to solve

The assert.yaml files in our integration test harness can assert that two files are the same or different.
This can fail arbitrarily for integration tests where order of events across streams are not guaranteed - for
example, when indexing to elastic search.

We should review file based assertions and improve accordingly. Lexicographic sorting by line
might work for the elastic case, but our mileage may vary as new connectors are onboarded.

Describe the solution you'd like

A more comprehensive suite of assertions that are less brittle from a CI perspective ( sorting, ordering )
or when running on the command line ( tests need a clean state or to be reentrant w.r.t assertion state ).

Notes

There is no urgency here. A workaround is to use a separate assertion for each line being asserted. Verbose but workable for now.

@darach darach added the enhancement New feature or request label Apr 12, 2024
@darach darach self-assigned this Apr 12, 2024
@darach
Copy link
Member Author

darach commented Apr 12, 2024

See #2482 where there is a workaround for lack of sorting/ordering in asserts for elastic-verify-gd integration test
The addition of empty file assertions makes the tremor new cli command test work in stale ( dev ) and fresh ( CI test run ) environments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant