Refactor Jenkinsfiles #13344
Refactor Jenkinsfiles #13344
Conversation
|
@mxnet-label-bot add [pr-awaiting-review] |
marcoabreu
force-pushed
the
split-jenkins-pipeline
branch
from
a4d5d69 to
cdb4b80
Compare
marcoabreu
added
pr-work-in-progress
marcoabreu
force-pushed
the
split-jenkins-pipeline
branch
from
28fee7f to
8cb25fa
Compare
marcoabreu
added
pr-awaiting-review
marcoabreu
force-pushed
the
split-jenkins-pipeline
branch
from
0305879 to
95dd04a
Compare
ci/jenkins/Jenkinsfile_edge
Outdated
| utils = load('ci/Jenkinsfile_utils.groovy') | ||
| custom_steps = load('ci/jenkins/Jenkins_steps.groovy') | ||
| } | ||
| utils.assign_node_labels(linux_cpu: 'mxnetlinux-cpu', linux_gpu: 'mxnetlinux-gpu', linux_gpu_p3: 'mxnetlinux-gpu-p3', windows_cpu: 'mxnetwindows-cpu', windows_gpu: 'mxnetwindows-gpu') |
There was a problem hiding this comment.
as discussed offline with @marcoabreu, it would be nice to abstract away the need to have windows node labeling in test files that are testing centos or arm, etc.
There was a problem hiding this comment.
As discussed offline with @marcoabreu, we could have a common failure handler for the Jenkins files that would make it more maintainable. This is left for a future work. LGTM.
marcoabreu
added
pr-awaiting-merge
| // Credit to https://plugins.jenkins.io/github | ||
| def get_repo_url() { | ||
| checkout scm | ||
| sh "git config --get remote.origin.url > .git/remote-url" |
There was a problem hiding this comment.
Why not .execute() writing a file doesn't seem the best way.
There was a problem hiding this comment.
There is no way to directly access shell output in Jenkins https://stackoverflow.com/questions/36507410/is-it-possible-to-capture-the-stdout-from-the-sh-dsl-command-in-the-pipeline
I could've used other methods, but that would then have triggered the security mechanisms of Jenkins due to the sandboxing mechanism. Execute (although I don't know exactly which function you're referring to) sounds like it would probably trigger that restriction.
There was a problem hiding this comment.
Ah, they've recently added a redirect parameter. I've replaced it at #13355. Thanks for the heads up!
This PR refactors our Jenkinsfiles that are used in the PR stage for the following reasons:
Test jobs to validate these changes are available at http://jenkins.mxnet-ci-dev.amazon-ml.com/view/test-marco-mxnet/
The status report would then look as follows:
So far, I have not refactored other pipelines. This would be done in a follow-up PR.