Skip to content

Commit

Permalink
docs: updates the docs and e2e
Browse files Browse the repository at this point in the history
This make changes to the developer and user documentation. And also
updates the k8s version used in the e2e test to a version where we have
an AMI available.

Signed-off-by: Richard Case <richard.case@outlook.com>
  • Loading branch information
richardcase committed Oct 14, 2024
1 parent d421a43 commit 1ec7f47
Show file tree
Hide file tree
Showing 4 changed files with 84 additions and 24 deletions.
3 changes: 1 addition & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,7 @@ cluster on AWS.

- Native Kubernetes manifests and API
- Manages the bootstrapping of VPCs, gateways, security groups and instances.
- Choice of Linux distribution among Amazon Linux 2, CentOS 7, Ubuntu and Flatcar
using [pre-baked AMIs][published_amis].
- Choice of Linux distribution using [pre-baked AMIs][published_amis].
- Deploys Kubernetes control planes into private subnets with a separate
bastion server.
- Doesn't use SSH for bootstrapping nodes.
Expand Down
78 changes: 65 additions & 13 deletions docs/book/src/development/amis.md
Original file line number Diff line number Diff line change
@@ -1,30 +1,82 @@
# Publish AMIs

Publishing new AMIs is currently a manual process but it will be automated in th every near future (see [this issue](https://github.com/kubernetes-sigs/cluster-api-provider-aws/issues/1982) for progress).
Publishing new AMIs is done via manually invoking a GitHub Actions workflow.

## Pre-reqs
> NOTE: the plan is to ultimately fully automate the process in the future (see [this issue](https://github.com/kubernetes-sigs/cluster-api-provider-aws/issues/1982) for progress).
- You must have admin access to the CNCF AWAS account used for the AMIs (819546954734)
> NOTE: there are some issues with the RHEL based images at present.
## Get build inputs

For a new Kubernetes version that you want to build an AMI for you will need to determine the following values:

| Input | Description |
| ----------------- | ----------- |
| kubernetes_semver | The semver version of k8s you want to build an AMI for. In format vMAJOR.MINOR.PATCH. |
| kubernetes_series | The release series for the Kubernetes version. In format vMAJOR.MINOR. |
| kubernetes_deb_version | The version of the debian package for the release. |
| kubernetes_rpm_version | The version of the rpm package for the release |
| kubernetes_cni_semver | The version of CNI to include. It needs to match the k8s release. |
| kubernetes_cni_deb_version | The version of the debian package for the CNI release to use |
| crictl_version | The vesion of the cri-tools package to install into the AMI |

You can determine these values directly or by looking at the publish debian apt repositories for the k8s release.

## Build

### Using GitHub Actions Workflow

To build the AMI using GitHub actions you must have write access to the CAPA repository (i.e. be a maintainer or part of release team).

To build the new version:

## Process
1. Got to the GitHub Action
2. Click the **Start Workflow** button
3. Fill in the details of the build
4. Click **Run**

### Manually

> **WARNING: the manual process should only be followed in exceptional circumstances.
To build manually you must have admin access to the CNCF AWS account used for the AMIs.

The steps to build manually are:

1. Clone [image-builder](https://github.com/kubernetes-sigs/image-builder)
2. Open a terminal
3. Set the AWS environment variables for the AMI account
3. Set the AWS environment variables for the CAPA AMI account
4. Change directory into `images/capi`
5. Install dependencies by running:
5. Create a new file called `vars.json` with the following content (substituing the values with the build inputs):

```json
{
"kubernetes_rpm_version": "<INSERT_INPUT_VALUE>",
"kubernetes_semver": "<INSERT_INPUT_VALUE>",
"kubernetes_series": "<INSERT_INPUT_VALUE>",
"kubernetes_deb_version": "<INSERT_INPUT_VALUE>",
"kubernetes_cni_semver": "<INSERT_INPUT_VALUE>",
"kubernetes_cni_deb_version": "<INSERT_INPUT_VALUE>",
"crictl_version": "<INSERT_INPUT_VALUE>"
}
```
6. Install dependencies by running:

```shell
make deps-ami
```

6. Build the AMIs using:
7. Build the AMIs using:

```shell
make build-ami-ubuntu-2004
make build-ami-ubuntu-2204
make build-ami-ubuntu-2404
make build-ami-flatcar
make build-ami-rhel-8
PACKER_VAR_FILES=vars.json make build-ami-ubuntu-2204
PACKER_VAR_FILES=vars.json make build-ami-ubuntu-2404
PACKER_VAR_FILES=vars.json make build-ami-flatcar
PACKER_VAR_FILES=vars.json make build-ami-rhel-8
```
> NOTE: there are some issues with the RHEL based images at present.
## Additional Information

- The AMIs are hosted in a CNCF owned AWS account (819546954734).
- The AWS resources that are needed to support the GitHub Actions workflow are created via terraform. Source is [here](https://github.com/kubernetes/k8s.io/tree/main/infra/aws/terraform/cncf-k8s-infra-aws-capa-ami).
- OIDC and IAM Roles are used to grant access via short lived credentials to the GitHub Action workflow instance when it runs.

21 changes: 15 additions & 6 deletions docs/book/src/topics/images/built-amis.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,28 @@
# Pre-built Kubernetes AMIs

New AMIs are built on a best effort basis when a new Kubernetes version is released. for each supported OS distribution and then published to supported regions.
New AMIs are built on a best effort basis when a new Kubernetes version is released for each supported OS distribution and then published to supported regions.

## AMI Publication Policy

- AMIs should only be used for non-production usage. For production environments we recommend that you build and maintain your own AMIs using the image-builder project.
- AMIs will only be published for the latest release series and 2 previous release series. For example, if the current release series is v1.30 then AMIs will only be published for v1.30, v1.29, v1.28.
- When there is a new k8s release series then any AMIs no longer covered by the previous point will be deleted. For example, when v1.31.0 is published then any AMIs for the v1.28 release series will be deleted.
- Existing AMIs are not updated for security fixes and it is recommended to always use the latest patch version for the Kubernetes version you want to run.

> NOTE: As the old community images where located in an AWS account that the project no longer has access to and because those AMIs have been automatically deleted we have started publishing images again from v1.29.9
## Finding AMIs

`clusterawsadm ami list` command lists pre-built reference AMIs by Kubernetes version, OS, or AWS region. See [clusterawsadm ami list](https://cluster-api-aws.sigs.k8s.io/clusterawsadm/clusterawsadm_ami_list.html) for details.

If you are using a version of clusterawsadm prior to v2.6.2 then you will need to explicitly specify the owner-id for the community account: `clusterawsadm ami list --owner-id 819546954734`.

> **Note:** These images are not updated for security fixes and it is recommended to always use the latest patch version for the Kubernetes version you want to run. For production environments, it is highly recommended to build and use your own custom images.
## Supported OS Distributions
- Amazon Linux 2 (amazon-2)
- Ubuntu (ubuntu-20.04, ubuntu-22.04, ubuntu-24.04)
- Centos (centos-7)
- Ubuntu (ubuntu-22.04, ubuntu-24.04)
- Flatcar (flatcar-stable)

> Note: Centos (centos-7) and Amazon Linux 2 (amazon-2) where supported but there are some issues with the AMI build that need fixing.
## Supported AWS Regions
- ap-northeast-1
- ap-northeast-2
Expand Down
6 changes: 3 additions & 3 deletions test/e2e/data/e2e_conf.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -177,8 +177,8 @@ variables:
# The following Kubernetes versions should be the latest versions with already published kindest/node images.
# This avoids building node images in the default case which improves the test duration significantly.
KUBERNETES_VERSION_MANAGEMENT: "v1.29.0"
KUBERNETES_VERSION: "v1.26.6"
KUBERNETES_VERSION_UPGRADE_TO: "v1.26.6"
KUBERNETES_VERSION: "v1.29.9"
KUBERNETES_VERSION_UPGRADE_TO: "v1.29.9"
KUBERNETES_VERSION_UPGRADE_FROM: "v1.25.12"
# Pre and post 1.23 Kubernetes versions are being used for CSI upgrade tests
PRE_1_23_KUBERNETES_VERSION: "v1.22.17"
Expand All @@ -190,7 +190,7 @@ variables:
AWS_NODE_MACHINE_TYPE: t3.large
AWS_MACHINE_TYPE_VCPU_USAGE: 2
AWS_SSH_KEY_NAME: "cluster-api-provider-aws-sigs-k8s-io"
CONFORMANCE_CI_ARTIFACTS_KUBERNETES_VERSION: "v1.26.6"
CONFORMANCE_CI_ARTIFACTS_KUBERNETES_VERSION: "v1.29.9"
CONFORMANCE_WORKER_MACHINE_COUNT: "5"
CONFORMANCE_CONTROL_PLANE_MACHINE_COUNT: "3"
ETCD_VERSION_UPGRADE_TO: "3.5.6-0"
Expand Down

0 comments on commit 1ec7f47

Please sign in to comment.