You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The assert.yaml files in our integration test harness can assert that two files are the same or different.
This can fail arbitrarily for integration tests where order of events across streams are not guaranteed - for
example, when indexing to elastic search.
We should review file based assertions and improve accordingly. Lexicographic sorting by line
might work for the elastic case, but our mileage may vary as new connectors are onboarded.
Describe the solution you'd like
A more comprehensive suite of assertions that are less brittle from a CI perspective ( sorting, ordering )
or when running on the command line ( tests need a clean state or to be reentrant w.r.t assertion state ).
Notes
There is no urgency here. A workaround is to use a separate assertion for each line being asserted. Verbose but workable for now.
The text was updated successfully, but these errors were encountered:
See #2482 where there is a workaround for lack of sorting/ordering in asserts for elastic-verify-gd integration test
The addition of empty file assertions makes the tremor new cli command test work in stale ( dev ) and fresh ( CI test run ) environments.
Describe the problem you are trying to solve
The
assert.yaml
files in our integration test harness can assert that two files are the same or different.This can fail arbitrarily for integration tests where order of events across streams are not guaranteed - for
example, when indexing to elastic search.
We should review file based assertions and improve accordingly. Lexicographic sorting by line
might work for the elastic case, but our mileage may vary as new connectors are onboarded.
Describe the solution you'd like
A more comprehensive suite of assertions that are less brittle from a CI perspective ( sorting, ordering )
or when running on the command line ( tests need a clean state or to be reentrant w.r.t assertion state ).
Notes
There is no urgency here. A workaround is to use a separate assertion for each line being asserted. Verbose but workable for now.
The text was updated successfully, but these errors were encountered: