Daint / Sarus CI #30

haampie · 2020-04-03T16:05:32Z

The pipeline definition is stored here https://gitlab.com/cscs-ci/eth-cscs/cosma-ci

Builds are done on a kubernetes cluster, which pushes docker images to https://gitlab.com/cscs-ci/eth-cscs/COSMA/container_registry
No tests are run on Daint yet, since I have to write the appropriate sarus run ... scripts.

CI should be triggered by bors, let's see if that works as well.

haampie · 2020-04-03T16:05:37Z

bors try

bors · 2020-04-03T16:05:38Z

🔒 Permission denied

Existing reviewers: click here to make haampie a reviewer

haampie · 2020-04-03T16:06:31Z

Ok... I don't have access 😅, @kabicm, can you either comment bors try or give me push access temporarily?

kabicm · 2020-04-03T16:35:07Z

bors try

bors · 2020-04-03T16:35:12Z

try

Configuration problem

bors.toml: not found

haampie · 2020-04-03T17:36:19Z

bors try

bors · 2020-04-03T17:56:23Z

try

Build succeeded

ci/gitlab/gitlab.com

haampie · 2020-04-03T18:02:56Z

So, what this did is:

it created a build environment with CUDA, MKL and MPICH here

and compiled COSMA for this PR and bundled it with its dependencies to a small docker image here

Try

docker run -it --rm registry.gitlab.com/cscs-ci/eth-cscs/cosma/deploy:9232a10d37da252ae5627826609dbcb1ef46ae9a /bin/bash

to see the contents.

then pulled this image on Daint here, but it's not running it yet. That should still be implemented here.

The second time the build environment is cached as long as you don't bump MKL, CUDA Toolkit or MPICH, so let's see if that actually works as expected.

bors try

bors · 2020-04-03T18:10:01Z

try

Build succeeded

ci/gitlab/gitlab.com

…happy about libmpi

haampie · 2020-04-07T08:16:42Z

bors try

bors · 2020-04-07T08:32:26Z

try

Build failed

ci/gitlab/gitlab.com

haampie · 2020-04-07T08:34:22Z

bors try

bors · 2020-04-07T08:47:57Z

try

Build failed

ci/gitlab/gitlab.com

haampie · 2020-04-07T09:33:25Z

Last attempt hopefully? 😅

bors try

bors · 2020-04-07T09:41:58Z

try

Build failed

ci/gitlab/gitlab.com

haampie · 2020-04-07T09:43:15Z

@kabicm @teonnik can you maybe comment on what's going wrong here?

https://gitlab.com/cscs-ci/eth-cscs/COSMA/-/jobs/501552983

kabicm · 2020-04-07T11:24:40Z

@haampie Hard to say, is this using the latest commit on master?

haampie · 2020-04-09T11:22:58Z

Currently I'm only building the GPU version. I could add a CPU version as well!

kabicm · 2020-04-09T19:21:58Z

docker/deploy/Dockerfile

+COPY . /COSMA
+
+RUN mkdir /COSMA/build && cd /COSMA/build && \
+    CC=mpicc CXX=mpicxx FC=mpif90 FCCPP=cpp cmake .. \


I think FC=mpif90 FCCPP=cpp are not necessary since we don't use fortran compilers.

haampie · 2020-04-09T20:05:49Z

now with CPU and GPU both:

bors try

bors · 2020-04-09T20:10:02Z

try

Build failed

ci/gitlab/gitlab.com

haampie · 2020-04-09T20:12:46Z

bors try

bors · 2020-04-09T20:14:58Z

try

Build failed

ci/gitlab/gitlab.com

haampie · 2020-04-09T20:15:14Z

Now with new env variables for tile sizes

bors try

bors · 2020-04-09T20:29:06Z

try

Build failed

ci/gitlab/gitlab.com

haampie · 2020-04-09T20:31:10Z

Forgot to recursively clone the repo; another try

bors try

bors · 2020-04-09T20:39:35Z

try

Build succeeded

ci/gitlab/gitlab.com

haampie · 2020-04-09T20:39:59Z

😌

kabicm · 2020-04-09T23:52:07Z

This looks great @haampie! If @teonnik agrees, I think this is ready for merging, I am really looking forward to it!

teonnik

Looks good to me! :)

CMakeLists.txt

tests/CMakeLists.txt

haampie · 2020-04-10T12:18:31Z

Thanks @teonnik :) I've pushed the changes

teonnik · 2020-04-10T17:49:03Z

bors merge

bors · 2020-04-10T17:57:47Z

Build succeeded

ci/gitlab/gitlab.com

haampie added 3 commits April 3, 2020 17:55

Ensure that RUNPATHS are set when installing binaries/tests

0a0ba32

Make tests part of bin/ when running make install

1ba724d

Add an initial build docker and deploy docker image (currently GPU only)

68a7fa1

Make bors happy

0ce5f19

bors bot added a commit that referenced this pull request Apr 3, 2020

Try #30:

9232a10

bors bot added a commit that referenced this pull request Apr 3, 2020

Try #30:

ec73b6d

teonnik requested review from kabicm and teonnik April 3, 2020 18:25

kabicm added the feature label Apr 3, 2020

Simplify binary bundling a bit and make sarus and the nvidia runtime …

a73dd37

…happy about libmpi

bors bot added a commit that referenced this pull request Apr 7, 2020

Try #30:

712598b

bors bot added a commit that referenced this pull request Apr 7, 2020

Try #30:

479b1b4

bors bot added a commit that referenced this pull request Apr 7, 2020

Try #30:

fa8eba3

kabicm reviewed Apr 9, 2020

View reviewed changes

haampie added 4 commits April 9, 2020 21:31

Remove unused cosma_deps and link against blas instead

dcf5d52

Make BLAS a docker build variable

411bc1a

Bump bundler to 0.1.5

a5132ea

Use a slightly smaller MKL package and remove unused variables

d9eaefa

bors bot added a commit that referenced this pull request Apr 9, 2020

Try #30:

1a6f34a

bors bot added a commit that referenced this pull request Apr 9, 2020

Try #30:

32b888f

bors bot added a commit that referenced this pull request Apr 9, 2020

Try #30:

4c9dba6

Add the ci repo as a submodule for convenience

ed4e9fc

bors bot added a commit that referenced this pull request Apr 9, 2020

Try #30:

98ccc8d

teonnik requested changes Apr 10, 2020

View reviewed changes

CMakeLists.txt Outdated Show resolved Hide resolved

tests/CMakeLists.txt Outdated Show resolved Hide resolved

haampie added 2 commits April 10, 2020 14:14

preserve relocatibility of install folder

91eec8e

Use GNUInstallDirs instead of hard-coded paths

d05c8ed

teonnik approved these changes Apr 10, 2020

View reviewed changes

bors bot merged commit c8f020a into eth-cscs:master Apr 10, 2020

dbsanfte mentioned this pull request Oct 19, 2025

Add BFloat16 support for AI/ML workloads #155

Draft

6 tasks

Uh oh!

Daint / Sarus CI #30

Daint / Sarus CI #30

Uh oh!

Conversation

haampie commented Apr 3, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

haampie commented Apr 3, 2020

Uh oh!

bors bot commented Apr 3, 2020

Uh oh!

haampie commented Apr 3, 2020

Uh oh!

kabicm commented Apr 3, 2020

Uh oh!

bors bot commented Apr 3, 2020

try

Configuration problem

Uh oh!

haampie commented Apr 3, 2020

Uh oh!

bors bot commented Apr 3, 2020

try

Build succeeded

Uh oh!

haampie commented Apr 3, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bors bot commented Apr 3, 2020

try

Build succeeded

Uh oh!

haampie commented Apr 7, 2020

Uh oh!

bors bot commented Apr 7, 2020

try

Build failed

Uh oh!

haampie commented Apr 7, 2020

Uh oh!

bors bot commented Apr 7, 2020

try

Build failed

Uh oh!

haampie commented Apr 7, 2020

Uh oh!

bors bot commented Apr 7, 2020

try

Build failed

Uh oh!

haampie commented Apr 7, 2020

Uh oh!

kabicm commented Apr 7, 2020

Uh oh!

haampie commented Apr 9, 2020

Uh oh!

kabicm Apr 9, 2020

Choose a reason for hiding this comment

Uh oh!

haampie commented Apr 9, 2020

Uh oh!

bors bot commented Apr 9, 2020

try

Build failed

Uh oh!

haampie commented Apr 9, 2020

Uh oh!

bors bot commented Apr 9, 2020

try

Build failed

Uh oh!

haampie commented Apr 9, 2020

Uh oh!

bors bot commented Apr 9, 2020

try

Build failed

Uh oh!

haampie commented Apr 9, 2020

Uh oh!

haampie commented Apr 3, 2020 •

edited

Loading

haampie commented Apr 3, 2020 •

edited

Loading

teonnik left a comment •

edited

Loading