Integrate Go s3secrets-helper into elastic-stack

[pda]

Integrating buildkite/elastic-ci-stack-s3-secrets-hooks#37 into elastic-stack.

CI:

• Build the s3secrets-helper binaries as CI steps (linux & windows)
• Upload binaries as artifacts
• Add them to the packer AMI builds, place them in PATH

Development:

• Build the s3secrets-helper binaries via Makefile
• Packer will pick them up from there.

[yob]

👀

[yob]

Neat!

I tested this (and buildkite/elastic-ci-stack-s3-secrets-hooks#37) as a blackbox by creating a new stack based on this branch and running some builds.

My secrets bucket looked like this:

aws s3 ls s3://<bucket-name> --recursive
2020-11-22 22:31:56         26 env
2020-11-22 22:32:22          0 gist-test/
2020-11-22 22:32:50         28 gist-test/env
2020-11-22 22:29:43       1678 private_ssh_key

My global env:

$ cat env 
#!/bin/sh

GLOBAL_ENV=foo

My pipeline env:

$ cat env 
#!/bin/sh

PIPELINE_ENV=bar

The build script looked like this:

#!/bin/bash

set -ux

echo $GLOBAL_ENV
echo $PIPELINE_ENV

The hook ran in 0 seconds (woo!), and did everything I expected:

The environment variables set in the env files were available when the script ran:

There's some things I didn't test, like ssh credentials (my test repo was checked out via http), and git-credentials. Still, my tests show there's no glaring errors and I reckon we should merge so we can start dogfooding it via our own builds.

A few of these changes will conflict with the arm support that's brewing in #758. That's OK though, I'm happy to rebase the arm support if this merges first.

[yob]

Oh. I should add that I also ran the same test on windows and everything worked as expected 👍