[CI] support sending a PR to each ecs logging agent when spec changes

[v1v]

What is this PR doing?

It adds a Jenkins pipeline executing on pushes to the master branch, with the following stages:

• Checkouts the SCM
• Uses a parallel execution to send PRs to each subscribed ecs agents, if and only if there are changes in the spec files. To subscribe an agent, simply add an entry in the .ci/.jenkins-loggers.yml file:

---
loggers:
  - REPO: "ecs-logging-java"
    SPEC_FILEPATH: "ecs-logging-core/src/test/resources/spec/spec.json"

• Executes a shell script per agent, this script should:

1. clone the agent repo
2. ensure the target directory exists
3. copy the spec.json file to the target directory
4. create a git commit with the changes
5. send a PR to the default branch of the agent repo with the changes

Why is important?

We want to push changes to downstream repos (the ecs agents) whenever a change on the specs is done. This way the repos are always up-to-date with the specs.

Issues

Closes #30
Requires elastic/apm-pipeline-library#936

[apmmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Build Cause: Started by timer

• • Start Time: 2021-02-05T04:03:00.653+0000

• Duration: 3 min 53 sec

• Commit: 730c89a

[trentm]

For now, and probably for the foreseeable future, the ecs-logging.git spec file is just the single "spec.json" file, unlike the apm-server specs.

Also, the two go repos appear to be using a different filename ("v1.json") rather than just using the same "spec.json" file basename. That means this code will need to adjust. Some possible ideas:

• change "SPEC_PATH" in .jenkins-loggers.yml to be the full file path, e.g.:

...
  - REPO: "ecs-dotnet"
    SPEC_PATH: "tests/Elastic.CommonSchema.Tests/Specs/spec.json"
  - REPO: "ecs-logging-go-logrus"
    SPEC_PATH: "internal/spec/v1.json"
...

• Consider dropping the filesType variables. I think it is unlikely that anything other than spec.json will be relevant for the foreseeable future.
• Consider changing filesPath to fileName or something to indicate it is just the single file.

[axw]

Also, the two go repos appear to be using a different filename ("v1.json") rather than just using the same "spec.json" file basename. That means this code will need to adjust. Some possible ideas:

If it's less work overall, I think we could also just rename v1.json to spec.json in the Go repos. @simitt any concerns about that?

[v1v]

I just added support to different names with c8205f5 we can change if any repo changes the name of those spec.json files

[trentm]

Have you been able to test this? I know it is probably hard to do so realistically.

[trentm]

Is this a commit message that is wanted?

[v1v]

This is the commit in the PR, therefore the person who squash and merge could potentially change the title of the PR. Since there was no specifications at the time of writing this, but using what it was already implemented for other similar projects.

I guess, this is something that can be potentially changed in the future with the message that the apm agents have agreed, do you think for the time being we can move forward?

[trentm]

do you think for the time being we can move forward?

Yup, definitely. Thanks.