[automation] update elastic stack version for testing 7.16.0-ef210289

[apmmachine]

What

Bump stack version with the latest one.

Further details

[start_time:Sun, 5 Dec 2021 05:14:11 GMT, release_branch:7.16, prefix:, end_time:Sun, 5 Dec 2021 11:24:06 GMT, manifest_version:2.0.0, version:7.16.0-SNAPSHOT, branch:7.16, build_id:7.16.0-ef210289, build_duration_seconds:22195]

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2021-12-08T09:25:39.079+0000

• Duration: 44 min 47 sec

• Commit: 8cd45af

Test stats 🧪

Test	Results
Failed	0
Passed	345
Skipped	9
Total	354

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[jsoriano]

There seems to be some changes in timestamp formats with timezones, but the generated timestamps are equivalent. This might be related to elastic/elasticsearch#80450.

I am pushing updated files to fix this.

[jsoriano]

This looks like an actual fix, the timestamp in the logs is:

2020-04-01T11:21:06,725+0200

That converted to UTC should be 2020-04-01T09:21:06.725Z.

cc @sayden in case you want to confirm that the pipeline of this fileset is right.

[mergify]

Automatically approving mergify

[jsoriano]

/test

[jsoriano]

/test

[jsoriano]

@nik9000 we have found in our test files that the format of some timestamps with timezones changed with a recent snapshot of ES, you can see the ones that changed in this PR. Do you think it could be related to elastic/elasticsearch#80450?

Most of the new timestamps are equivalent the old ones, but there is one that seems to be fixed now: #29278 (comment)

[nik9000]

Do you think it could be related to elastic/elasticsearch#80450?

I don't think it's related but I'm happy to help figure out what changed. What are you testing? It looks like it's importing some logs from https://github.com/elastic/beats/blob/7.16/x-pack/filebeat/module/cisco/asa/test/sample.log and then fetching them with, like, a scroll? Are the things in https://github.com/elastic/beats/blob/7.16/x-pack/filebeat/module/cisco/asa/test/sample.log-expected.json a fetched _source? If so I'd look at pipelines or even logstash. I don't really know what's going on here.

[jsoriano]

I don't think it's related but I'm happy to help figure out what changed.

No worries, I was asking more out of curiosity in case something came to mind.

What are you testing?

These tests install a pipeline, ingest some logs using filebeat and check that the indexed documents match what is in the expected files. They are used to test the pipelines included in filebeat modules.

The new timestamps seem to be ok because they represent the same time (for example the change from "2018-04-15T09:34:34.000-04:00" to "2018-04-15T11:34:34.000-02:00").

What intrigues me is this change from "2020-04-01T13:21:06.725Z" to "2020-04-01T09:21:06.725Z", these are not equivalent timestamps, but the new one seems to be correct. In the logs this one is represented as 2020-04-01T11:21:06,725+0200.

So I wonder if there has been some fix in Elasticsearch, or if we have something weird in this pipeline.

Looking at how this is parsed in the pipeline, it looks a bit tricky, first the timestamp is parsed with formats with and without the timestamp:

  - date:
      if: ctx.elasticsearch.audit['@timestamp'] != null && ctx.event.timezone != null
      field: elasticsearch.audit.@timestamp
      target_field: elasticsearch.audit.@timestamp
      formats:
        - yyyy-MM-dd'T'HH:mm:ss,SSS
        - yyyy-MM-dd'T'HH:mm:ss,SSSZ
      timezone: "{{ event.timezone }}"
      ignore_failure: true

And then it is parsed again as ISO8601:

  - date:
      field: elasticsearch.audit.@timestamp
      target_field: "@timestamp"
      formats:
        - ISO8601
      ignore_failure: true

[nik9000]

I'm guessing its actually elastic/elasticsearch#63876.

[nik9000]

For reference I got this by running git blame ./modules/ingest-common/src/main/java/org/elasticsearch/ingest/common/DateProcessor.java --ignore-revs-file .git-blame-ignore-revs and looking for recent changes. There weren't any but I saw that a lot happens in DateFormat so I ran git blame ./modules/ingest-common/src/main/java/org/elasticsearch/ingest/common/DateFormat.java --ignore-revs-file .git-blame-ignore-revs which pointed me to the commit from that PR.

[jsoriano]

Thanks Nik! This change looks more likely, yes. It could be then that something has been actually fixed for this tricky pipeline.