`exit 1` Even though commands passed

[Sushisource]

The plugin is reliably exiting 1, and hence failing my runs, with rather long runtimes. I have a 4 hour test which triggers this every time.

My pipeline definition looks like:

agents:
  queue: 'default'
  docker: '*'

steps:
  - label: 'nightly-test'
    timeout_in_minutes: 360
    concurrency: 1 # Only allow one nightly build at a time
    concurrency_group: 'node-sdk/nightly'
    commands:
      - '.buildkite/docker/build.sh'
      - 'npm run ci-nightly'
      - 'echo success'
    artifact_paths:
      - 'docker-compose-logs/*'
    plugins:
      - docker-compose#v3.8.0:
          run: test-container
          upload-container-logs: always
          config:
            - .buildkite/docker/docker-compose.yaml
            - .buildkite/docker/docker-compose-ci.yaml

I know the commands work fine because I see the success line get echoed. The build output ends with:

success
🚨 Error: The command exited with status 1
user command error: The plugin docker-compose command hook exited with status 1

Whereas builds that have failed for legitimate reasons look like:

⚠️ Failed to run command, exited with 2, run params:
run --name buildkitec14d1406904d4db98fa70cefd10a4e75_test-container_build_250 --rm test-container /bin/sh -e -c .buildkite/docker/build.sh
npm run ci-load
🚨 Error: The command exited with status 2
user command error: The plugin docker-compose command hook exited with status 2

The "failed to run command" part being missing says to me the plugin isn't mistakenly thinking my command failed. It thinks it passed, and then exits 1 anyway.

I haven't dug into plugin code yet but this is preventing me from using the plugin for these longer running tests.

[toote]

Hi @Sushisource, sorry for the delay in getting back to you.

This was a real head-scratcher... and an actual bug 😱.

Apparently, the issue is that the when using an array for commands the buildkite agent creates a script with all the commands passed on to execute. This plugin doesn't do that and just passes through the BUILDKITE_COMMAND variable which is basically a very long string with newlines and it is the source of all your woes.

There are two possible workarounds:
1- move your commands to a script in your repository and use that as your step's command
2- do not to use the array version of commands in the step and change it to use a single string for the command (command1; command2; command3) or create the corresponding plugin configuration in the docker command format.

[toote]

Ok, after a lot of research on the matter this has been noted and discussed in the past (see #59 and #186).

I will add changes to the documentation and a warning in the output of the run step to ensure that this doesn't catch people off-guard.

[holylander]

I also faced the same issue and wanted to add my two cents here:

basically, the docker-compose plugin will join the command array elements using a ', so, in case you need to run multiple commands, rather than

              [...]
              mount-buildkite-agent: true
              command: [
                  ".buildkite/scripts/script1.sh",
                  ".buildkite/scripts/script2.sh",
                  "echo 'finished",
                ]

you can apply the below syntax in a single command:

              [...]
              mount-buildkite-agent: true
              command: [
                  "
                  .buildkite/scripts/script1.sh;
                  .buildkite/scripts/script2.sh;
                  echo 'finished'
                  ",
                ]

You can also get the same result replacing the ; by and &&.
Eventually, you may also need to leverage the shell plugin option, depending on which machine the BK agent is running.