Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add docs on updating the different components #957

Merged
merged 16 commits into from
Nov 22, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions docs/updating-agent.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Updating the Agent

The `buildkite-agent` is built in to the AMIs by the Packer build. The agent
binary is downloaded from download.buildkite.com.

Once you have [released](https://github.com/buildkite/agent/blob/master/RELEASE.md) an updated
version of the agent, you can incorporate it into the Elastic CI Stack
for AWS template.

See https://github.com/buildkite/elastic-ci-stack-for-aws/pull/935 for an
example of updating the buildkite-agent.

1. Create a new branch
1. Update and commit a change to the Packer install scripts for [Linux](#linux) and [Windows](#windows) for the new version
1. Push your branch and open a pull request
1. Wait for CI to go green
1. Merge

## Linux

Update the `AGENT_VERSION` variable in [`packer/linux/scripts/install-buildkite-agent.sh`](packer/linux/scripts/install-buildkite-agent.sh)
to change which version is installed during the Packer build for the Linux AMI.

## Windows

Update the `AGENT_VERSION` variable in [`packer/windows/scripts/install-buildkite-agent.ps1`](packer/windows/scripts/install-buildkite-agent.ps1)
to change which version is installed during the Packer build for the Windows
AMI.
29 changes: 29 additions & 0 deletions docs/updating-buildkite-agent-scaler.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Updating buildkite-agent-scaler

The [buildkite-agent-scaler](https://buildkite.com/buildkite/buildkite-agent-scaler)
is brought in to the Elastic CI Stack for AWS template using the AWS
Serverless Application Repository.

Once you have [released](https://github.com/buildkite/buildkite-agent-scaler/blob/master/RELEASE.md)
an updated version, you can incorporate it into the Elastic CI Stack for AWS
template.

keithduncan marked this conversation as resolved.
Show resolved Hide resolved
See https://github.com/buildkite/elastic-ci-stack-for-aws/pull/955 for an
example of updating the buildkite-agent-scaler.

1. Create a new branch
1. In the [`templates/aws-stack.yml`](templates/aws-stack.yml) update the `Autoscaling` resource’s `SemanticVersion` property to the newly released version.
1. Push your branch and open a pull request
1. Wait for CI to pass
1. If needed for testing, create a stack with the branch’s published template to verify functionality
1. In the Buildkite build, get the template URL from the build annotation
1. In the AWS Console, launch CloudFormation
1. Create a new stack, select new resources
1. Supply the template URL from the build annotation for the Amazon S3 URL field
1. Supply a Buildkite Agent token in the `BuildkiteAgentToken`
or `BuildkiteAgentTokenParameterStorePath` parameters
1. Supply a queue in the `BuildkiteQueue` parameter
1. Create the stack and wait for it to complete
1. Verify that instances are booted in response to Buildkite jobs on your queue
1. Delete the stack
1. Merge your pull request
41 changes: 41 additions & 0 deletions docs/updating-docker.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Updating Docker and docker-compose

Docker and docker-compose are built into the AMIs by the Packer build.

See https://github.com/buildkite/elastic-ci-stack-for-aws/pull/954 for an
example of updating Docker and docker-compose for Linux and Windows.

1. Create a new branch
1. Update and commit a change to the Packer install scripts for [Linux](#linux) and [Windows](#windows) for the new version(s)
1. Push your branch and open a pull request
1. Wait for CI to go green
1. Merge

## Linux

Packer installs Docker, docker-compose, and `docker buildx` using the [`packer/linux/scripts/install-docker.sh`](packer/linux/scripts/install-docker.sh)
script. Update the `DOCKER_VERSION` variable in this file to change which
version of Docker is installed. Update the `DOCKER_COMPOSE_VERSION` variable to
change which version of docker-compose is installed. Update the
`DOCKER_BUILDX_VERSION` variable to change which version of `docker buildx` is installed.

Binary releases are downloaded directly from Docker and GitHub.

## Windows

Packer installs Docker, and `docker-compose` using the [`packer/windows/scripts/install-docker.ps1`](packer/windows/scripts/install-docker.ps1)
script.

[MicrosoftDockerProvider](https://github.com/OneGet/MicrosoftDockerProvider)
is used to source Docker packages. The list of available packages that it
installs can be found at https://dockermsft.azureedge.net/dockercontainer/DockerMsftIndex.json.

Update the `docker_version` variable in `install-docker.ps1` to one of the
available package versions to change which version of Docker is installed.

The `choco` package manager is used to install `docker-compose`. See the
[chocolatey package manager version history](https://community.chocolatey.org/packages/docker-compose#versionhistory)
for a list of available versions.

Update the `docker_compose_version` variable in `install-docker.ps1` to change
which version of `docker-compose` is installed.
30 changes: 30 additions & 0 deletions docs/updating-secrets.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Updating elastic-ci-stack-s3secrets-hooks

The elastic-ci-stack-s3secrets-hooks are included in the AMIs by the Packer
build. The hooks are copied in directly from the submodule, the binaries are
downloaded from the GitHub release.

keithduncan marked this conversation as resolved.
Show resolved Hide resolved
Once you have [released](https://github.com/buildkite/elastic-ci-stack-s3-secrets-hooks/blob/master/RELEASE.md)
an updated version of the s3secrets-hooks, you can incorporate it into the
Elastic CI Stack for AWS template.

See https://github.com/buildkite/elastic-ci-stack-for-aws/pull/956 for an
example of updating elastic-ci-stack-s3secrets-hooks.

1. Create a new branch
1. Update the `plugins/secrets` git submodule and `.gitmodules` file for the new tag
1. `git submodule init`
1. `cd plugins/secrets && git checkout v2.1.x`
1. Commit the change to the submodule pin and `.gitmodules`
1. Update and commit a change to the Packer install scripts for [Linux](#linux) and [Windows](#windows) to use the new version
keithduncan marked this conversation as resolved.
Show resolved Hide resolved
1. Push your branch and open a pull request
1. Wait for CI to go green
1. Merge

## Linux

Update `S3_SECRETS_HELPER_VERSION` in [`packer/linux/scripts/install-s3secrets-helper.sh`](packer/linux/scripts/install-s3secrets-helper.sh)

## Windows

Update `S3_SECRETS_HELPER_VERSION` in [`packer/windows/scripts/install-s3secrets-helper.ps1`](packer/windows/scripts/install-s3secrets-helper.ps1)