Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: build packages in parallel #17723

Merged
merged 22 commits into from
Apr 17, 2020
Merged

feat: build packages in parallel #17723

Changes from 1 commit
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
2a47768
feat: build packages in parallel
kuisathaverat Apr 15, 2020
866f166
fix: unstable on a stage ignored
kuisathaverat Apr 15, 2020
eb1a613
fix: unstable on a stage ignored
kuisathaverat Apr 15, 2020
2dd22e9
fix: make beats in parallel by architecture
kuisathaverat Apr 15, 2020
009ae72
fix: use correct PLATFORMS format
kuisathaverat Apr 15, 2020
ab7f6c6
fix: synstax error
kuisathaverat Apr 15, 2020
733080e
fix: method code too large! again
kuisathaverat Apr 15, 2020
1345540
fix: use error step
kuisathaverat Apr 15, 2020
d29bbca
fix: add goos method
kuisathaverat Apr 15, 2020
4727387
fix: use only one axis
kuisathaverat Apr 16, 2020
a9c9863
fix: path for scripts
kuisathaverat Apr 16, 2020
d9902b7
fix: store artifects
kuisathaverat Apr 16, 2020
d6cd7a6
fix: add linux platforms
kuisathaverat Apr 16, 2020
f75bd82
test: use checkout always
kuisathaverat Apr 16, 2020
efab4c4
fix: search for packages also on the root build folder
kuisathaverat Apr 16, 2020
e6b1617
fix: PLATFORMS variable
kuisathaverat Apr 16, 2020
67ac9c5
fix: remove defaults
kuisathaverat Apr 16, 2020
3aca138
feta: push docker images for the CI use
kuisathaverat Apr 16, 2020
f8995aa
fix: typo
kuisathaverat Apr 16, 2020
adfa160
fix: script worng
kuisathaverat Apr 17, 2020
f122b1f
fix: the other way around
kuisathaverat Apr 17, 2020
8f54245
list Docker images
kuisathaverat Apr 17, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 36 additions & 42 deletions .ci/packaging.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,10 @@ pipeline {
BASE_DIR = 'src/github.com/elastic/beats'
JOB_GCS_BUCKET = 'beats-ci-artifacts'
JOB_GCS_CREDENTIALS = 'beats-ci-gcs-plugin'
SNAPSHOT = "true"
}
options {
timeout(time: 2, unit: 'HOURS')
timeout(time: 3, unit: 'HOURS')
buildDiscarder(logRotator(numToKeepStr: '20', artifactNumToKeepStr: '20', daysToKeepStr: '30'))
timestamps()
ansiColor('xterm')
Expand All @@ -35,56 +36,49 @@ pipeline {
}
}
stage('Linux'){
agent { label 'ubuntu && immutable' }
options { skipDefaultCheckout() }
when {
beforeAgent true
expression { return params.linux }
}
environment {
SNAPSHOT = "true"
PLATFORMS = "+linux/armv7 +linux/ppc64le +linux/s390x +linux/mips64"
HOME = "${env.WORKSPACE}"
}
steps {
deleteDir()
unstash 'source'
dir("${BASE_DIR}"){
sh(label: 'Release', script: './dev-tools/jenkins_release.sh')
matrix {
agent { label 'ubuntu && immutable' }
axes {
axis {
name 'PLATFORMS'
values '+linux/armv7', '+linux/ppc64le', '+linux/s390x', '+linux/mips64', 'darwin'
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I'm missing something, but don't we build windows packages?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not from the betas-ci, we only have this jobs https://beats-ci.elastic.co/job/elastic+beats+master+package/
I've check https://github.com/elastic/beats/blob/master/dev-tools/mage/platforms.go#L29-L70 and there is a bunch of platforms we do not build on the beats-ci, Should we build them?

Copy link
Contributor Author

@kuisathaverat kuisathaverat Apr 15, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

checking the logs, I think does no matter what you put on PLATFORMS env var you always build all the packages

This is a log where PLATFORMS is +linux/s390x
https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/pipelines/packaging/branches/PR-17723/runs/1/nodes/155/log/?start=0

but after build all the cross-compile Docker images I think it is compiling packages that are not requested.

[2020-04-15T10:42:45.027Z] >> Building using: cmd='build/mage-linux-amd64 buildGoDaemon', env=[CC=s390x-linux-gnu-gcc, CXX=s390x-linux-gnu-g++, GOARCH=s390x, GOARM=, GOOS=linux, PLATFORM_ID=linux-s390x]
[2020-04-15T10:42:45.027Z] >> Building using: cmd='build/mage-linux-amd64 buildGoDaemon', env=[CC=aarch64-linux-gnu-gcc, CXX=aarch64-linux-gnu-g++, GOARCH=arm64, GOARM=, GOOS=linux, PLATFORM_ID=linux-arm64]
[2020-04-15T10:42:45.027Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=o64-clang, CXX=o64-clang++, GOARCH=amd64, GOARM=, GOOS=darwin, PLATFORM_ID=darwin-amd64]
[2020-04-15T10:42:45.027Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=x86_64-w64-mingw32-gcc, CXX=x86_64-w64-mingw32-g++, GOARCH=amd64, GOARM=, GOOS=windows, PLATFORM_ID=windows-amd64]
[2020-04-15T10:43:26.771Z] >> Building using: cmd='build/mage-linux-amd64 buildGoDaemon', env=[CC=gcc, CXX=g++, GOARCH=amd64, GOARM=, GOOS=linux, PLATFORM_ID=linux-amd64]
[2020-04-15T10:43:26.771Z] >> Building using: cmd='build/mage-linux-amd64 buildGoDaemon', env=[CC=gcc, CXX=g++, GOARCH=386, GOARM=, GOOS=linux, PLATFORM_ID=linux-386]
[2020-04-15T10:43:29.276Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=aarch64-linux-gnu-gcc, CXX=aarch64-linux-gnu-g++, GOARCH=arm64, GOARM=, GOOS=linux, PLATFORM_ID=linux-arm64]
[2020-04-15T10:43:29.276Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=gcc, CXX=g++, GOARCH=386, GOARM=, GOOS=linux, PLATFORM_ID=linux-386]
[2020-04-15T10:46:05.836Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=gcc, CXX=g++, GOARCH=amd64, GOARM=, GOOS=linux, PLATFORM_ID=linux-amd64]
[2020-04-15T10:46:19.465Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=i686-w64-mingw32-gcc, CXX=i686-w64-mingw32-g++, GOARCH=386, GOARM=, GOOS=windows, PLATFORM_ID=windows-386]
[2020-04-15T10:47:15.806Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=s390x-linux-gnu-gcc, CXX=s390x-linux-gnu-g++, GOARCH=s390x, GOARM=, GOOS=linux, PLATFORM_ID=linux-s390x]
[2020-04-15T10:51:47.176Z] >> Building using: cmd='build/mage-linux-amd64 buildGoDaemon', env=[CC=s390x-linux-gnu-gcc, CXX=s390x-linux-gnu-g++, GOARCH=s390x, GOARM=, GOOS=linux, PLATFORM_ID=linux-s390x]
[2020-04-15T10:51:47.176Z] >> Building using: cmd='build/mage-linux-amd64 buildGoDaemon', env=[CC=aarch64-linux-gnu-gcc, CXX=aarch64-linux-gnu-g++, GOARCH=arm64, GOARM=, GOOS=linux, PLATFORM_ID=linux-arm64]
[2020-04-15T10:51:47.176Z] >> Building using: cmd='build/mage-linux-amd64 buildGoDaemon', env=[CC=gcc, CXX=g++, GOARCH=amd64, GOARM=, GOOS=linux, PLATFORM_ID=linux-amd64]
[2020-04-15T10:51:47.177Z] >> Building using: cmd='build/mage-linux-amd64 buildGoDaemon', env=[CC=gcc, CXX=g++, GOARCH=386, GOARM=, GOOS=linux, PLATFORM_ID=linux-386]
[2020-04-15T10:51:48.682Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=x86_64-w64-mingw32-gcc, CXX=x86_64-w64-mingw32-g++, GOARCH=amd64, GOARM=, GOOS=windows, PLATFORM_ID=windows-amd64]
[2020-04-15T10:51:48.989Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=o64-clang, CXX=o64-clang++, GOARCH=amd64, GOARM=, GOOS=darwin, PLATFORM_ID=darwin-amd64]
[2020-04-15T10:51:49.289Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=gcc, CXX=g++, GOARCH=amd64, GOARM=, GOOS=linux, PLATFORM_ID=linux-amd64]
[2020-04-15T10:51:49.577Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=gcc, CXX=g++, GOARCH=386, GOARM=, GOOS=linux, PLATFORM_ID=linux-386]
[2020-04-15T10:55:11.890Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=aarch64-linux-gnu-gcc, CXX=aarch64-linux-gnu-g++, GOARCH=arm64, GOARM=, GOOS=linux, PLATFORM_ID=linux-arm64]
[2020-04-15T10:55:39.158Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=s390x-linux-gnu-gcc, CXX=s390x-linux-gnu-g++, GOARCH=s390x, GOARM=, GOOS=linux, PLATFORM_ID=linux-s390x]
[2020-04-15T10:55:47.588Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=i686-w64-mingw32-gcc, CXX=i686-w64-mingw32-g++, GOARCH=386, GOARM=, GOOS=windows, PLATFORM_ID=windows-386]
[2020-04-15T11:00:17.233Z] >> Building using: cmd='build/mage-linux-amd64 buildGoDaemon', env=[CC=gcc, CXX=g++, GOARCH=amd64, GOARM=, GOOS=linux, PLATFORM_ID=linux-amd64]
[2020-04-15T11:00:17.233Z] >> Building using: cmd='build/mage-linux-amd64 buildGoDaemon', env=[CC=s390x-linux-gnu-gcc, CXX=s390x-linux-gnu-g++, GOARCH=s390x, GOARM=, GOOS=linux, PLATFORM_ID=linux-s390x]
[2020-04-15T11:00:17.233Z] >> Building using: cmd='../../heartbeat/build/mage-linux-amd64 golangCrossBuild', env=[CC=x86_64-w64-mingw32-gcc, CXX=x86_64-w64-mingw32-g++, GOARCH=amd64, GOARM=, GOOS=windows, PLATFORM_ID=windows-amd64]
[2020-04-15T11:00:17.233Z] >> Building using: cmd='../../heartbeat/build/mage-linux-amd64 golangCrossBuild', env=[CC=o64-clang, CXX=o64-clang++, GOARCH=amd64, GOARM=, GOOS=darwin, PLATFORM_ID=darwin-amd64]
[2020-04-15T11:00:19.419Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=x86_64-w64-mingw32-gcc, CXX=x86_64-w64-mingw32-g++, GOARCH=amd64, GOARM=, GOOS=windows, PLATFORM_ID=windows-amd64]
[2020-04-15T11:00:19.420Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=o64-clang, CXX=o64-clang++, GOARCH=amd64, GOARM=, GOOS=darwin, PLATFORM_ID=darwin-amd64]
[2020-04-15T11:04:10.993Z] >> Building using: cmd='../../heartbeat/build/mage-linux-amd64 golangCrossBuild', env=[CC=gcc, CXX=g++, GOARCH=386, GOARM=, GOOS=linux, PLATFORM_ID=linux-386]
[2020-04-15T11:04:15.760Z] >> Building using: cmd='../../heartbeat/build/mage-linux-amd64 golangCrossBuild', env=[CC=gcc, CXX=g++, GOARCH=amd64, GOARM=, GOOS=linux, PLATFORM_ID=linux-amd64]
[2020-04-15T11:04:47.868Z] >> Building using: cmd='../../heartbeat/build/mage-linux-amd64 golangCrossBuild', env=[CC=aarch64-linux-gnu-gcc, CXX=aarch64-linux-gnu-g++, GOARCH=arm64, GOARM=, GOOS=linux, PLATFORM_ID=linux-arm64]
[2020-04-15T11:05:04.160Z] >> Building using: cmd='../../heartbeat/build/mage-linux-amd64 golangCrossBuild', env=[CC=s390x-linux-gnu-gcc, CXX=s390x-linux-gnu-g++, GOARCH=s390x, GOARM=, GOOS=linux, PLATFORM_ID=linux-s390x]
[2020-04-15T11:08:25.855Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=gcc, CXX=g++, GOARCH=386, GOARM=, GOOS=linux, PLATFORM_ID=linux-386]
[2020-04-15T11:08:47.841Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=gcc, CXX=g++, GOARCH=amd64, GOARM=, GOOS=linux, PLATFORM_ID=linux-amd64]
[2020-04-15T11:09:34.544Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=aarch64-linux-gnu-gcc, CXX=aarch64-linux-gnu-g++, GOARCH=arm64, GOARM=, GOOS=linux, PLATFORM_ID=linux-arm64]
[2020-04-15T11:09:47.741Z] >> Building using: cmd='../../heartbeat/build/mage-linux-amd64 golangCrossBuild', env=[CC=i686-w64-mingw32-gcc, CXX=i686-w64-mingw32-g++, GOARCH=386, GOARM=, GOOS=windows, PLATFORM_ID=windows-386]
[2020-04-15T11:11:54.243Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=i686-w64-mingw32-gcc, CXX=i686-w64-mingw32-g++, GOARCH=386, GOARM=, GOOS=windows, PLATFORM_ID=windows-386]
[2020-04-15T11:12:01.802Z] >> Building using: cmd='build/mage-linux-amd64 buildGoDaemon', env=[CC=gcc, CXX=g++, GOARCH=386, GOARM=, GOOS=linux, PLATFORM_ID=linux-386]
[2020-04-15T11:12:04.365Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=s390x-linux-gnu-gcc, CXX=s390x-linux-gnu-g++, GOARCH=s390x, GOARM=, GOOS=linux, PLATFORM_ID=linux-s390x]
[2020-04-15T11:13:26.020Z] >> Building using: cmd='build/mage-linux-amd64 buildGoDaemon', env=[CC=aarch64-linux-gnu-gcc, CXX=aarch64-linux-gnu-g++, GOARCH=arm64, GOARM=, GOOS=linux, PLATFORM_ID=linux-arm64]
[2020-04-15T11:17:35.398Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=s390x-linux-gnu-gcc, CXX=s390x-linux-gnu-g++, GOARCH=s390x, GOARM=, GOOS=linux, PLATFORM_ID=linux-s390x]
[2020-04-15T11:18:24.276Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=gcc, CXX=g++, GOARCH=amd64, GOARM=, GOOS=linux, PLATFORM_ID=linux-amd64]
[2020-04-15T11:18:24.276Z] >> Building using: cmd='../../journalbeat/build/mage-linux-amd64 golangCrossBuild', env=[CC=gcc, CXX=g++, GOARCH=386, GOARM=, GOOS=linux, PLATFORM_ID=linux-386]
[2020-04-15T11:18:24.276Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=gcc, CXX=g++, GOARCH=386, GOARM=, GOOS=linux, PLATFORM_ID=linux-386]
[2020-04-15T11:20:42.369Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=aarch64-linux-gnu-gcc, CXX=aarch64-linux-gnu-g++, GOARCH=arm64, GOARM=, GOOS=linux, PLATFORM_ID=linux-arm64]
[2020-04-15T11:22:49.461Z] >> Building using: cmd='../../journalbeat/build/mage-linux-amd64 golangCrossBuild', env=[CC=s390x-linux-gnu-gcc, CXX=s390x-linux-gnu-g++, GOARCH=s390x, GOARM=, GOOS=linux, PLATFORM_ID=linux-s390x]
[2020-04-15T11:23:43.333Z] >> Building using: cmd='../../journalbeat/build/mage-linux-amd64 golangCrossBuild', env=[CC=gcc, CXX=g++, GOARCH=amd64, GOARM=, GOOS=linux, PLATFORM_ID=linux-amd64]
[2020-04-15T11:24:41.950Z] >> Building using: cmd='../../journalbeat/build/mage-linux-amd64 golangCrossBuild', env=[CC=aarch64-linux-gnu-gcc, CXX=aarch64-linux-gnu-g++, GOARCH=arm64, GOARM=, GOOS=linux, PLATFORM_ID=linux-arm64]
[2020-04-15T11:24:52.326Z] >> Building using: cmd='build/mage-linux-amd64 buildGoDaemon', env=[CC=s390x-linux-gnu-gcc, CXX=s390x-linux-gnu-g++, GOARCH=s390x, GOARM=, GOOS=linux, PLATFORM_ID=linux-s390x]
[2020-04-15T11:24:54.897Z] >> Building using: cmd='build/mage-linux-amd64 buildGoDaemon', env=[CC=gcc, CXX=g++, GOARCH=386, GOARM=, GOOS=linux, PLATFORM_ID=linux-386]
[2020-04-15T11:24:57.437Z] >> Building using: cmd='build/mage-linux-amd64 buildGoDaemon', env=[CC=gcc, CXX=g++, GOARCH=amd64, GOARM=, GOOS=linux, PLATFORM_ID=linux-amd64]
[2020-04-15T11:24:59.987Z] >> Building using: cmd='build/mage-linux-amd64 buildGoDaemon', env=[CC=aarch64-linux-gnu-gcc, CXX=aarch64-linux-gnu-g++, GOARCH=arm64, GOARM=, GOOS=linux, PLATFORM_ID=linux-arm64]
[2020-04-15T11:31:30.088Z] >> Building using: cmd='build/mage-linux-amd64 buildGoDaemon', env=[CC=s390x-linux-gnu-gcc, CXX=s390x-linux-gnu-g++, GOARCH=s390x, GOARM=, GOOS=linux, PLATFORM_ID=linux-s390x]
[2020-04-15T11:31:30.088Z] >> Building using: cmd='build/mage-linux-amd64 buildGoDaemon', env=[CC=aarch64-linux-gnu-gcc, CXX=aarch64-linux-gnu-g++, GOARCH=arm64, GOARM=, GOOS=linux, PLATFORM_ID=linux-arm64]
[2020-04-15T11:31:30.088Z] >> Building using: cmd='build/mage-linux-amd64 buildGoDaemon', env=[CC=gcc, CXX=g++, GOARCH=amd64, GOARM=, GOOS=linux, PLATFORM_ID=linux-amd64]
[2020-04-15T11:31:30.088Z] >> Building using: cmd='build/mage-linux-amd64 buildGoDaemon', env=[CC=gcc, CXX=g++, GOARCH=386, GOARM=, GOOS=linux, PLATFORM_ID=linux-386]
[2020-04-15T11:31:31.895Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=x86_64-w64-mingw32-gcc, CXX=x86_64-w64-mingw32-g++, GOARCH=amd64, GOARM=, GOOS=windows, PLATFORM_ID=windows-amd64]
[2020-04-15T11:31:31.895Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=gcc, CXX=g++, GOARCH=amd64, GOARM=, GOOS=linux, PLATFORM_ID=linux-amd64]
[2020-04-15T11:31:32.179Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=s390x-linux-gnu-gcc, CXX=s390x-linux-gnu-g++, GOARCH=s390x, GOARM=, GOOS=linux, PLATFORM_ID=linux-s390x]
[2020-04-15T11:31:32.179Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=o64-clang, CXX=o64-clang++, GOARCH=amd64, GOARM=, GOOS=darwin, PLATFORM_ID=darwin-amd64]
[2020-04-15T11:36:53.973Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=gcc, CXX=g++, GOARCH=386, GOARM=, GOOS=linux, PLATFORM_ID=linux-386]
[2020-04-15T11:37:15.932Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=aarch64-linux-gnu-gcc, CXX=aarch64-linux-gnu-g++, GOARCH=arm64, GOARM=, GOOS=linux, PLATFORM_ID=linux-arm64]
[2020-04-15T11:37:24.636Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=i686-w64-mingw32-gcc, CXX=i686-w64-mingw32-g++, GOARCH=386, GOARM=, GOOS=windows, PLATFORM_ID=windows-386]
[2020-04-15T11:43:27.014Z] >> Building using: cmd='../../packetbeat/build/mage-linux-amd64 golangCrossBuild', env=[CC=i686-w64-mingw32-gcc, CXX=i686-w64-mingw32-g++, GOARCH=386, GOARM=, GOOS=windows, PLATFORM_ID=windows-386]
[2020-04-15T11:43:27.014Z] >> Building using: cmd='build/mage-linux-amd64 buildGoDaemon', env=[CC=gcc, CXX=g++, GOARCH=386, GOARM=, GOOS=linux, PLATFORM_ID=linux-386]
[2020-04-15T11:43:27.279Z] >> Building using: cmd='build/mage-linux-amd64 buildGoDaemon', env=[CC=aarch64-linux-gnu-gcc, CXX=aarch64-linux-gnu-g++, GOARCH=arm64, GOARM=, GOOS=linux, PLATFORM_ID=linux-arm64]
[2020-04-15T11:43:27.279Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=i686-w64-mingw32-gcc, CXX=i686-w64-mingw32-g++, GOARCH=386, GOARM=, GOOS=windows, PLATFORM_ID=windows-386]
[2020-04-15T11:43:27.539Z] >> Building using: cmd='build/mage-linux-amd64 buildGoDaemon', env=[CC=gcc, CXX=g++, GOARCH=amd64, GOARM=, GOOS=linux, PLATFORM_ID=linux-amd64]
[2020-04-15T11:43:27.539Z] >> Building using: cmd='build/mage-linux-amd64 buildGoDaemon', env=[CC=s390x-linux-gnu-gcc, CXX=s390x-linux-gnu-g++, GOARCH=s390x, GOARM=, GOOS=linux, PLATFORM_ID=linux-s390x]
[2020-04-15T11:45:49.679Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=x86_64-w64-mingw32-gcc, CXX=x86_64-w64-mingw32-g++, GOARCH=amd64, GOARM=, GOOS=windows, PLATFORM_ID=windows-amd64]
[2020-04-15T11:46:02.163Z] >> Building using: cmd='../../packetbeat/build/mage-linux-amd64 golangCrossBuild', env=[CC=x86_64-w64-mingw32-gcc, CXX=x86_64-w64-mingw32-g++, GOARCH=amd64, GOARM=, GOOS=windows, PLATFORM_ID=windows-amd64]
[2020-04-15T11:48:24.606Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=o64-clang, CXX=o64-clang++, GOARCH=amd64, GOARM=, GOOS=darwin, PLATFORM_ID=darwin-amd64]
[2020-04-15T11:48:24.606Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=gcc, CXX=g++, GOARCH=amd64, GOARM=, GOOS=linux, PLATFORM_ID=linux-amd64]
[2020-04-15T11:48:24.606Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=s390x-linux-gnu-gcc, CXX=s390x-linux-gnu-g++, GOARCH=s390x, GOARM=, GOOS=linux, PLATFORM_ID=linux-s390x]
[2020-04-15T11:48:24.606Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=gcc, CXX=g++, GOARCH=386, GOARM=, GOOS=linux, PLATFORM_ID=linux-386]
[2020-04-15T11:52:41.540Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=aarch64-linux-gnu-gcc, CXX=aarch64-linux-gnu-g++, GOARCH=arm64, GOARM=, GOOS=linux, PLATFORM_ID=linux-arm64]
[2020-04-15T11:52:49.115Z] >> Building using: cmd='../../packetbeat/build/mage-linux-amd64 golangCrossBuild', env=[CC=s390x-linux-gnu-gcc, CXX=s390x-linux-gnu-g++, GOARCH=s390x, GOARM=, GOOS=linux, PLATFORM_ID=linux-s390x]
[2020-04-15T11:52:57.588Z] >> Building using: cmd='../../packetbeat/build/mage-linux-amd64 golangCrossBuild', env=[CC=gcc, CXX=g++, GOARCH=386, GOARM=, GOOS=linux, PLATFORM_ID=linux-386]
[2020-04-15T11:53:02.372Z] >> Building using: cmd='../../packetbeat/build/mage-linux-amd64 golangCrossBuild', env=[CC=gcc, CXX=g++, GOARCH=amd64, GOARM=, GOOS=linux, PLATFORM_ID=linux-amd64]
[2020-04-15T11:57:14.075Z] >> Building using: cmd='../../packetbeat/build/mage-linux-amd64 golangCrossBuild', env=[CC=aarch64-linux-gnu-gcc, CXX=aarch64-linux-gnu-g++, GOARCH=arm64, GOARM=, GOOS=linux, PLATFORM_ID=linux-arm64]
[2020-04-15T11:58:15.176Z] >> Building using: cmd='../../packetbeat/build/mage-linux-amd64 golangCrossBuild', env=[CC=o64-clang, CXX=o64-clang++, GOARCH=amd64, GOARM=, GOOS=darwin, PLATFORM_ID=darwin-amd64]

Copy link
Contributor Author

@kuisathaverat kuisathaverat Apr 15, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, checking the output of one of the old jobs, the PLATFORMS env var it is ignored always build all the packages

https://beats-ci.elastic.co/job/elastic+beats+7.x+multijob-package-linux/125/console

}
}
publishPackages()
}
}
stage('Mac OS'){
agent { label 'macosx' }
options { skipDefaultCheckout() }
when {
beforeAgent true
expression { return params.linux }
}
environment {
KEYCHAIN = "/var/lib/jenkins/Library/Keychains/Elastic.keychain-db"
SNAPSHOT = "true"
PLATFORMS = "darwin"
APPLE_SIGNING_ENABLED = "true"
HOME = "${env.WORKSPACE}"
}
steps {
deleteDir()
unstash 'source'
withMaskEnv( vars: [
[var: "KEYCHAIN_PASS", password: getVaultSecret(secret: "secret/jenkins-ci/macos-codesign-keychain").data.password],
]){
dir("${BASE_DIR}"){
sh(label: 'Release', script: './dev-tools/jenkins_release.sh')
stages {
stage('Package'){
environment {
HOME = "${env.WORKSPACE}"
}
steps {
deleteDir()
unstash 'source'
release()
publishPackages()
}
}
publishPackages()
}
}
}
}
}

def release(){
dir("${BASE_DIR}"){
if(env.PLATFORMS == 'darwin' && params.macos){
withMaskEnv( vars: [
[var: "KEYCHAIN_PASS", password: getVaultSecret(secret: "secret/jenkins-ci/macos-codesign-keychain").data.password],
[var: "KEYCHAIN", password: "/var/lib/jenkins/Library/Keychains/Elastic.keychain-db"],
[var: "APPLE_SIGNING_ENABLED", password: "true"],
]){
sh(label: 'Release', script: './dev-tools/jenkins_release.sh')
}
} else if (params.linux){
sh(label: 'Release', script: './dev-tools/jenkins_release.sh')
kuisathaverat marked this conversation as resolved.
Show resolved Hide resolved
}
}
}

def publishPackages(){
googleStorageUpload(bucket: "gs://${JOB_GCS_BUCKET}/snapshots",
credentialsId: "${JOB_GCS_CREDENTIALS}",
Expand Down