Integrate large curated data sets into the documentation tests (e.g. Kibana sample data)

[lcawl]

Currently the examples in the Elasticsearch documentation are tested using the functionality described here: https://github.com/elastic/elasticsearch/tree/master/docs#snippet-testing

The data that is supplied to the examples is typically a very small data set defined in setup scripts in https://github.com/elastic/elasticsearch/blob/master/docs/build.gradle
Some of our features, however, need large data sets to generate interesting results (or any results at all).

Ideally our documentation testing system should be able to pull in large data sets (e.g. Kibana sample data or other public demo data) that customers also have access to. Ideally those data sets should be easy for folks to find and run on their own deployments. These data sets should also be curated such that they provide meaningful results for one or more features. Otherwise, the API examples just prove the API runs but don’t return anything realistic that customers can explore and learn from.

[elasticmachine]

Pinging @elastic/es-docs (:Docs)

[jrodewig]

Aside from curating the actual data sets, this is currently possible and done. The accounts.json data set used in the Getting Started section is pretty large at 237KB:

https://github.com/elastic/elasticsearch/blob/master/docs/src/test/resources/accounts.json

It's pulled into the tests here:
https://github.com/elastic/elasticsearch/blob/master/docs/build.gradle#L279

We'd have to be mindful of curating a lot of these and ensuring our test file stays synced with the data set provided to the user though.

[lcawl]

One of the appealing aspects of the Kibana sample data set is that it contains three sets that contain timestamps, which are required for many of our machine learning examples and (unlike previous data sets we've uploaded for customer use) those timestamps are relative to when you install the data. Thus I don't have to go into the publicly available data sets every year and level up the timestamps manually.

[elasticmachine]

Pinging @elastic/es-core-infra (:Core/Infra/Build)