Open
Description
Is your feature request related to a problem?
Currently, Doctest module depends on backend and use the result set for comparison directly. Specifically, whenever there is filtering operation (SQL WHERE
clause or PPL where
command), sorting by _doc
field is added automatically. This is supposed to be special case and moved to client side.
What solution would you like?
Sort the result rows in consistent way before formatting and comparison. This is similar as what our comparison test framework does. This approach may be favored over the alternative as follows.
What alternatives have you considered?
- Add
ORDER BY
clause orsort
command to each SQL/PPL query to make sure consistent order in result set. This may complicate the sample query and thus make it unclear to users. - Remove the default sorting and fix all doctest manually. However, the result maybe still not deterministic and the test may fail if the result rows is returned in different order.
Do you have any additional context?
public void pushDown(QueryBuilder query) {
...
if (sourceBuilder.sorts() == null) {
sourceBuilder.sort(DOC_FIELD_NAME, ASC); // Make sure consistent order
}
}
Doctest depends on the logic above:
ppl_cmd.process('source=accounts | where account_number=1 or gender="F" | fields account_number, gender')Expected:
fetched rows / total rows = 2/2
+------------------+----------+
| account_number | gender |
|------------------+----------|
| 1 | M |
| 13 | F |
+------------------+----------+
Got:
fetched rows / total rows = 2/2
+------------------+----------+
| account_number | gender |
|------------------+----------|
| 13 | F |
| 1 | M |
+------------------+----------+