Skip to content

Commit 9b68ea4

Browse files
blakerouseapmmachineapmmachinephrdner
authored
[v2] Merge July 27th main into v2 feature branch (#789)
* [Automation] Update elastic stack version to 8.4.0-40cff009 for testing (#557) Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co> * [Automation] Update elastic stack version to 8.4.0-5e6770b1 for testing (#564) Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co> * Fix regression and use comma separated values (#560) Fix regression from #509 * Change in Jenkinsfile will trigger k8s run (#568) * [Automation] Update elastic stack version to 8.4.0-da5a1c6d for testing (#573) Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co> * Add `@metadata.input_id` and `@metadata.stream_id` when injecting streams (#527) These 2 value are going to be used in the shipper to identify where an event came from in order to apply processors accordingly. Also, added test cases for the processor to verify the change and updated test cases with the new processor. * Add filemod times to contents of diagnostics collect command (#570) * Add filemod times to contents of diagnostics collect command Add filemod times to the files and directories in the zip archive. Log files (and sub dirs) will use the modtime returned by the fileinfo for the source. Others will use the timestamp from when the zip is created. * Fix linter * [Automation] Update elastic stack version to 8.4.0-b13123ee for testing (#581) Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co> * Fix Agent upgrade 8.2->8.3 (#578) * Fix Agent upgrade 8.2->8.3 * Improve the upgrade encryption handling. Add .yml files cleanup. * Rollback ActionUpgrade to action_id, add MarkerActionUpgrade adapter struct for marker serialization compatibility * Update containerd (#577) * [Automation] Update elastic stack version to 8.4.0-4fe26f2a for testing (#591) Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co> * Set explicit ExitTimeOut for MacOS agent launchd plist (#594) * Set explicit ExitTimeOut for MacOS agent launchd plist * [Automation] Update elastic stack version to 8.4.0-2e32a640 for testing (#599) Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co> * ci: enable build notifications as GitHub issues (#595) * status identifies failing component, fleet gateway may report degraded, liveness endpoint added (#569) * Add liveness endpoint Add /liveness route to metrics server. This route will report the status from pkg/core/status. fleet-gateway will now report a degraded state if a checkin fails. This may not propogate to fleet-server as a failed checkin means communications between the agent and the server are not working. It may also lead to the server reporting degraded for up to 30s (fleet-server polling time) when teh agent is able to successfully connect. * linter fix * add nolint direcrtive * Linter fix * Review feedback, add doc strings * Rename noop controller file to _test file * [Automation] Update elastic stack version to 8.4.0-722a7d79 for testing (#607) Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co> * ci: enable flaky test detector (#605) * [Automation] Update elastic stack version to 8.4.0-210dd487 for testing (#620) Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co> * mergify: remove backport automation for non active branches (#615) * chore: use elastic-agent profile to run the E2E tests (#610) * [Automation] Update elastic stack version to 8.4.0-a6aa9f3b for testing (#631) Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co> * add macros pointing to new agent's repo and fix old macro calls (#458) * Add mount of /etc/machine-id for managed Agent in k8s (#530) * Set hostPID=true for managed agent in k8s (#528) * Set hostPID=true for managed agent in k8s * Add comment on hostPID. * [Automation] Update elastic stack version to 8.4.0-86cc80f3 for testing (#648) Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co> * Update elastic-agent-libs version: includes restriction on default VerificationMode to `full` (#521) * update version * mage fmt update * update dependency * update changelog * redact sensitive information in diagnostics collect command (#566) * Support Cloudbeat regex input type (#638) * support input type with regex * Update supported.go * Changing the regex to support backward compatible * Disable flaky test download test (#641) * [Automation] Update elastic stack version to 8.4.0-3d206b5d for testing (#656) Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co> * [Automation] Update elastic stack version to 8.4.0-3ad82aa8 for testing (#661) Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co> * jjbb: exclude allowed branches, tags and PRs (#658) cosmetic change in the description and boolean based * Update elastic-agent-project-board.yml (#649) * ci: fix labels that clashes with the Orka workers (#659) * [Automation] Update elastic stack version to 8.4.0-03bd6f3f for testing (#668) Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co> * [Automation] Update elastic stack version to 8.4.0-533f1e30 for testing (#675) Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co> * Osquerybeat: Fix osquerybeat is not running with logstash output (#674) * [Automation] Update elastic stack version to 8.4.0-d0a4da44 for testing (#684) Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co> * [Automation] Update elastic stack version to 8.4.0-dd98ded4 for testing (#703) Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co> * [Automation] Update elastic stack version to 8.4.0-164d9a10 for testing (#705) Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co> * Add missing license headers (#711) * [Automation] Update elastic stack version to 8.4.0-00048b66 for testing (#713) Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co> * Allow - in eql variable names (#710) * fix to allow dashes in variable names in EQL expressions extend eql to allow the '-' char to appear in variable names, i.e., ${data.some-var} and additional test cases to eql, the transpiler, and the k8s provider to verify this works. Note that the bug was caused by the EQL limitation, the otehr test cases were added when attempting to find it. * Regenerate grammer with antlr 4.7.1, add CHANGELOG * Fix linter issue * Fix typo * Fix transpiler to allow : in dynamic variables. (#680) Fix transpiler regex to allow ':' characters in dynamic variables so that users can input "${dynamic.lookup|'fallback.here'}". Co-authored-by: Aleksandr Maus <aleksandr.maus@elastic.co> * Fix for the filebeat spec file picking up packetbeat inputs (#700) * Reproduce filebeat picking up packetbeat inputs * Filebeat: filter inputs as first input transform. Move input filtering to be the first input transformation that occurs in the filebeat spec file. Fixes #427. * Update changelog. * [Automation] Update elastic stack version to 8.4.0-3cd57abb for testing (#724) Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co> * [Automation] Update elastic stack version to 8.4.0-a324b98b for testing (#727) Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co> * ci: run on MacOS12 (#696) * [Automation] Update elastic stack version to 8.4.0-31315ca3 for testing (#732) Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co> * fix typo on package command (#734) This commit fixes the typo in the package command on the README.md. * Allow / to be used in variable names (#718) * Allow the / character to be used in variable names. Allow / to be used in variable names from dynamic providers and eql expressions. Ensure that k8s providers can provide variables with slashes in their names. * run antlr4 * Fix tests * Fix Elastic Agent non-fleet broken upgrade between 8.3.x releases (#701) * Fix Elastic Agent non-fleet broken upgrade between 8.3.x releases * Migrates vault directory on linux and windows to the top directory of the agent, so it can be shared without needing the upgrade handler call, like for example with side-by-side install/upgrade from .rpm/.deb * Extended vault to allow read-only open, useful when the vault at particular location needs to be only read not created. * Correct the typo in the log messages * Update lint flagged function comment with 'unused', was flagged with 'deadcode' on the previous run * Address code review feedback * Add missing import for linux utz * Change vault path from Top() to Config(), this a better location, next to fleet.enc based on the install/upgrade testing with .rpm/.deb installs * Fix the missing state migration for .rpm/.deb upgrade. The post install script now performs the migration and creates the symlink after that. * Fix typo in the postinstall script * Update the vault migration code, add the agent configuration match check with the agent secret * [Automation] Update elastic stack version to 8.4.0-31269fd2 for testing (#746) Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co> * wrap errors and fix some docs typo and convention (#743) * automate the ironbank docker context generation (#679) * Update README.md Adding M1 variable to export to be able to build AMD images * fix flaky (#730) * Add filestream ID on standalone kubernetes manifest (#742) This commit add unique IDs for the filestream inputs used by the Kubernetes integration in the Elastic-Agent standalone Kubernetes configuration/manifest file. * Alter github action to run on different OSs (#769) Alter the linter action to run on different OSs instead of on linux with the $GOOS env var. * [Automation] Update elastic stack version to 8.4.0-d058e92f for testing (#771) Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co> * elastic-agent manifests: add comments; add cloudnative team as a codeowner for the k8s manifests (#708) * managed elastic-agent: add comments; add cloudnative team as a codeowner for the k8s manifests Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co> * add comments to the standalone elastic-agent, similar to the documentation we have https://www.elastic.co/guide/en/fleet/current/running-on-kubernetes-standalone.html Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co> * Apply suggestions from code review Co-authored-by: Michael Katsoulis <michaelkatsoulis88@gmail.com> Co-authored-by: Andrew Gizas <andreas.gkizas@elastic.co> * remove comment for FLEET_ENROLLMENT_TOKEN; use Needed everywhere instead of Required Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co> * rephrase regarding accessing kube-state-metrics when used third party tools, like kube-rbac-proxy Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co> * run make check Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co> * keep manifests in sync to pass ci check Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co> * add info on where to find FLEET_URL and FLEET_ENROLLMENT_TOKEN Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co> * add links to elastic-agent documentation Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co> * update comment on FLEET_ENROLLMENT_TOKEN Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co> Co-authored-by: Michael Katsoulis <michaelkatsoulis88@gmail.com> Co-authored-by: Andrew Gizas <andreas.gkizas@elastic.co> * [Elastic-Agent] Added source uri reloading (#686) * Update will cleanup unneeded artifacts. (#752) * Update will cleanup unneeded artifacts. The update process will cleanup unneeded artifacts. When an update starts all artifacts that do not have the current version number in it's name will be removed. If artifact retrieval fails, downloaded artifacts are removed. On a successful upgrade, all contents of the downloads dir will be removed. * Clean up linter warnings * Wrap errors * cleanup tests * Fix passed version * Use os.RemoveAll * ci: propagate e2e-testing errors (#695) * [Release] add-backport-next (#784) * Update go.sum. * Fix upgrade. * Fix the upgrade artifact reload. * Fix lint in coordinator. Co-authored-by: apmmachine <58790750+apmmachine@users.noreply.github.com> Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co> Co-authored-by: Pier-Hugues Pellerin <phpellerin@gmail.com> Co-authored-by: Denis Rechkunov <denis.rechkunov@elastic.co> Co-authored-by: Michel Laterman <82832767+michel-laterman@users.noreply.github.com> Co-authored-by: Aleksandr Maus <aleksandr.maus@elastic.co> Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com> Co-authored-by: Anderson Queiroz <anderson.queiroz@elastic.co> Co-authored-by: Daniel Araujo Almeida <daniel-almeida@users.noreply.github.com> Co-authored-by: Mariana Dima <mariana@elastic.co> Co-authored-by: ofiriro3 <ofiriro3@Gmail.com> Co-authored-by: Julien Lind <julien.lind@elastic.co> Co-authored-by: Craig MacKenzie <craig.mackenzie@elastic.co> Co-authored-by: Tiago Queiroz <tiago.queiroz@elastic.co> Co-authored-by: Pierre HILBERT <pierre.hilbert@elastic.co> Co-authored-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co> Co-authored-by: Michael Katsoulis <michaelkatsoulis88@gmail.com> Co-authored-by: Andrew Gizas <andreas.gkizas@elastic.co> Co-authored-by: Michal Pristas <michal.pristas@gmail.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
1 parent ff667df commit 9b68ea4

File tree

69 files changed

+3896
-1558
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

69 files changed

+3896
-1558
lines changed

.ci/Jenkinsfile

Lines changed: 14 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ pipeline {
5050
dir("${BASE_DIR}"){
5151
setEnvVar('ONLY_DOCS', isGitRegionMatch(patterns: [ '.*\\.(asciidoc|md)' ], shouldMatchAll: true).toString())
5252
setEnvVar('PACKAGING_CHANGES', isGitRegionMatch(patterns: [ '(^dev-tools/packaging/.*|.ci/Jenkinsfile)' ], shouldMatchAll: false).toString())
53-
setEnvVar('K8S_CHANGES', isGitRegionMatch(patterns: [ '(^deploy/kubernetes/.*|^version/docs/version.asciidoc)' ], shouldMatchAll: false).toString())
53+
setEnvVar('K8S_CHANGES', isGitRegionMatch(patterns: [ '(^deploy/kubernetes/.*|^version/docs/version.asciidoc|.ci/Jenkinsfile)' ], shouldMatchAll: false).toString())
5454
}
5555
}
5656
}
@@ -79,7 +79,7 @@ pipeline {
7979
axes {
8080
axis {
8181
name 'PLATFORM'
82-
values 'ubuntu-20.04 && immutable', 'aarch64', 'windows-2016 && windows-immutable', 'windows-2022 && windows-immutable', 'darwin && orka && x86_64'
82+
values 'ubuntu-20.04 && immutable', 'aws && aarch64', 'windows-2016 && windows-immutable', 'windows-2022 && windows-immutable', 'macos12 && x86_64'
8383
}
8484
}
8585
stages {
@@ -164,7 +164,7 @@ pipeline {
164164
}
165165
}
166166
environment {
167-
ARCH = "${PLATFORM.equals('aarch64') ? 'arm64' : 'amd64'}"
167+
ARCH = "${PLATFORM.contains('aarch64') ? 'arm64' : 'amd64'}"
168168
DEV = true
169169
EXTERNAL = true
170170
}
@@ -175,7 +175,7 @@ pipeline {
175175
withMageEnv(){
176176
dir("${BASE_DIR}"){
177177
withPackageEnv("${PLATFORM}") {
178-
cmd(label: 'Go package', script: 'mage package')
178+
cmd(label: 'Go package', script: 'mage package ironbank')
179179
uploadPackagesToGoogleBucket(
180180
credentialsId: env.JOB_GCS_EXT_CREDENTIALS,
181181
repo: env.REPO,
@@ -219,7 +219,7 @@ pipeline {
219219
axes {
220220
axis {
221221
name 'K8S_VERSION'
222-
values "v1.24.0, v1.23.6, v1.22.9, v1.21.12"
222+
values "v1.24.0", "v1.23.6", "v1.22.9", "v1.21.12"
223223
}
224224
}
225225
stages {
@@ -251,18 +251,23 @@ pipeline {
251251
}
252252
steps {
253253
// TODO: what's the testMatrixFile to be used if any
254-
runE2E(testMatrixFile: '',
254+
runE2E(testMatrixFile: '.ci/.e2e-tests-for-elastic-agent.yaml',
255255
beatVersion: "${env.BEAT_VERSION}-SNAPSHOT",
256256
elasticAgentVersion: "${env.BEAT_VERSION}-SNAPSHOT",
257257
gitHubCheckName: "e2e-tests",
258258
gitHubCheckRepo: env.REPO,
259-
gitHubCheckSha1: env.GIT_BASE_COMMIT)
259+
gitHubCheckSha1: env.GIT_BASE_COMMIT,
260+
propagate: true,
261+
wait: true)
260262
}
261263
}
262264
}
263265
post {
264266
cleanup {
265-
notifyBuildResult(prComment: true)
267+
notifyBuildResult(prComment: true,
268+
analyzeFlakey: !isTag(), jobName: getFlakyJobName(withBranch: (isPR() ? env.CHANGE_TARGET : env.BRANCH_NAME)),
269+
githubIssue: isBranch() && currentBuild.currentResult != "SUCCESS",
270+
githubLabels: 'Team:Elastic-Agent-Control-Plane')
266271
}
267272
}
268273
}
@@ -274,7 +279,7 @@ def isCodeCoverageEnabled() {
274279

275280
def withPackageEnv(platform, Closure body) {
276281
if (isUnix()) {
277-
if (platform.contains('macosx')) {
282+
if (isDarwin()) {
278283
withPackageDarwinEnv() {
279284
body()
280285
}

.ci/jobs/elastic-agent-mbp.yml

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
- job:
33
name: "elastic-agent/elastic-agent-mbp"
44
display-name: elastic-agent
5-
description: "POC to isolate elastic agent from beats"
5+
description: "Elastic agent"
66
project-type: multibranch
77
script-path: .ci/Jenkinsfile
88
scm:
@@ -12,6 +12,7 @@
1212
discover-pr-forks-trust: permission
1313
discover-pr-origin: merge-current
1414
discover-tags: true
15+
head-filter-regex: '(main|7\.17|8\.\d+|PR-.*|v\d+\.\d+\.\d+)'
1516
notification-context: 'fleet-ci'
1617
repo: elastic-agent
1718
repo-owner: elastic
@@ -39,4 +40,4 @@
3940
timeout: 100
4041
timeout: '15'
4142
use-author: true
42-
wipe-workspace: 'True'
43+
wipe-workspace: true

.github/CODEOWNERS

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,4 @@
1-
# Team responsable for Fleet Server
1+
# Team responsible for Fleet Server
22
* @elastic/elastic-agent-control-plane
3+
4+
/deploy/kubernetes @elastic/obs-cloudnative-monitoring

.github/workflows/elastic-agent-project-board.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ jobs:
1414
with:
1515
headers: '{"GraphQL-Features": "projects_next_graphql"}'
1616
query: |
17-
mutation add_to_project($projectid:String!,$contentid:String!) {
17+
mutation add_to_project($projectid:[ID!]!,$contentid:ID!) {
1818
updateIssue(input: {id:$contentid, projectIds:$projectid}) {
1919
clientMutationId
2020
}

.github/workflows/golangci-lint.yml

Lines changed: 2 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -14,18 +14,10 @@ jobs:
1414
golangci:
1515
strategy:
1616
matrix:
17-
include:
18-
- GOOS: windows
19-
- GOOS: linux
20-
- GOOS: darwin
17+
os: [ ubuntu-latest, macos-latest, windows-latest ]
2118
name: lint
22-
runs-on: ubuntu-latest
19+
runs-on: ${{ matrix.os }}
2320
steps:
24-
- name: Echo details
25-
env:
26-
GOOS: ${{ matrix.GOOS }}
27-
run: echo Go GOOS=$GOOS
28-
2921
- uses: actions/checkout@v2
3022

3123
# Uses Go version from the repository.
@@ -38,8 +30,6 @@ jobs:
3830
go-version: "${{ steps.goversion.outputs.version }}"
3931

4032
- name: golangci-lint
41-
env:
42-
GOOS: ${{ matrix.GOOS }}
4333
uses: golangci/golangci-lint-action@v2
4434
with:
4535
# Optional: version of golangci-lint to use in form of v1.2 or v1.2.3 or `latest` to use the latest version

.gitignore

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -60,4 +60,3 @@ pkg/component/fake/fake
6060

6161
# VSCode
6262
/.vscode
63-

.mergify.yml

Lines changed: 9 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -181,55 +181,42 @@ pull_request_rules:
181181
labels:
182182
- "backport"
183183
title: "[{{ destination_branch }}](backport #{{ number }}) {{ title }}"
184-
- name: backport patches to 8.0 branch
185-
conditions:
186-
- merged
187-
- label=backport-v8.0.0
188-
actions:
189-
backport:
190-
assignees:
191-
- "{{ author }}"
192-
branches:
193-
- "8.0"
194-
labels:
195-
- "backport"
196-
title: "[{{ destination_branch }}](backport #{{ number }}) {{ title }}"
197-
- name: backport patches to 8.1 branch
184+
- name: backport patches to 8.2 branch
198185
conditions:
199186
- merged
200-
- label=backport-v8.1.0
187+
- label=backport-v8.2.0
201188
actions:
202189
backport:
203190
assignees:
204191
- "{{ author }}"
205192
branches:
206-
- "8.1"
193+
- "8.2"
207194
labels:
208195
- "backport"
209196
title: "[{{ destination_branch }}](backport #{{ number }}) {{ title }}"
210-
- name: backport patches to 8.2 branch
197+
- name: backport patches to 8.3 branch
211198
conditions:
212199
- merged
213-
- label=backport-v8.2.0
200+
- label=backport-v8.3.0
214201
actions:
215202
backport:
216203
assignees:
217204
- "{{ author }}"
218205
branches:
219-
- "8.2"
206+
- "8.3"
220207
labels:
221208
- "backport"
222209
title: "[{{ destination_branch }}](backport #{{ number }}) {{ title }}"
223-
- name: backport patches to 8.3 branch
210+
- name: backport patches to 8.4 branch
224211
conditions:
225212
- merged
226-
- label=backport-v8.3.0
213+
- label=backport-v8.4.0
227214
actions:
228215
backport:
229216
assignees:
230217
- "{{ author }}"
231218
branches:
232-
- "8.3"
219+
- "8.4"
233220
labels:
234221
- "backport"
235222
title: "[{{ destination_branch }}](backport #{{ number }}) {{ title }}"

0 commit comments

Comments
 (0)