This module manages an Amazon Web Services bastion EC2 instance and its Auto Scaling Group, Instance Profile / Role, CloudWatch Log Group, Security Group, and SSH Key Pair. The Auto Scaling Group will recreate the bastion if there is an issue with the EC2 instance or the availability zone where it is running.
The EC2 UserData assumes the Ubuntu operating system, which is configured as follows:
- Packages are updated, and the bastion is rebooted if required.
- If SSH hostkeys are present in the configurable S3 bucket and path, they are copied to the bastion to retain its previous SSH identity. If there are no host keys in S3, the current keys are copied there.
- The CloudWatch Logs Agent is installed and configured to ship logs from these files:
/var/log/syslog/var/log/auth.log
- A host record, named using the
bastion_namemodule input, is added to a configurable Route53 DNS zone for the current public IP address of the bastion. This happens via a script configured to run each time the bastion boots. - Automatic updates are configured, using a configurable time to reboot, and the email address to receive errors.
- By default sudo access is removed from the ubuntu user unless the
remove_root_accessinput is set to "false." - Additional EC2 User Data can be executed, for one-off configuration not included in this module.
- Additional users can be created and populated with their own
authorized_keysfile.
To proxy SSH connections to Kubernetes nodes through the bastion, add configuration like the following to the top of the ssh_config file. Replace the following information with your own values:
domain.comwith the same domain name that was specified as a Route53 zone ID in the instance of the bastion Terraform module. This is the domain name where the bastions host record will have been created during boot./path/to/ssh/private/keywith the path to your SSH private key file.172.20.*.*with the VPC CIDR.
# Define options to be used when connecting to the bastion.
host bastion.domain.com
IdentityFile /path/to/ssh/private/key
IdentitiesOnly yes
User ubuntu
# Use the bastion to proxy SSH connections to IPs in the VPC
# You can also add a DNS wildcard to the end of the next line
# if you use DNS resolution to access Kubernetes nodes.
host 172.20.*.*
ProxyCommand ssh -i /path/to/ssh/private/key ubuntu@bastion.domain.com -W %h:%p
You can now SSH directly to IP addresses within 172.20.0.0/16, and your connection will be proxied through the bastion.
You can proxy access to a private Kubernetes API through the bastion, instead of using a VPN.
Run the following to forward connections from port 8443 on your workstation, to a private Kubernetes API, - replace api.clustername.domain.com with your private API hostname, and bastion.domain.com with the hostname of your bastion:
ssh -L 8443:api.clustername.domain.com:443 ubuntu@bastion.domain.com
In another terminal tab, edit your KubeConfig and replace api.clustername.domain.com with 127.0.0.1:8443 in the server line for your private cluster.
With the above, as long as the SSH proxy connection remains active, you can use kubectl to access your private Kubernetes cluster. Close the SSH connection in the other terminal tab to stop proxying to the private API.
The sshuttle tool uses NAT redirect firewall rules to proxy access to a network over a bastion. This is useful to connect to multiple ports on multiple hosts without maintaining a lot of SSH forwarding.
The bastion already has Python installed, which sshuttle requires to be on the bastion. Once you have installed sshuttle on your workstation, use the following to redirect access to your VPC CIDR over the bastion - replacing bastion.domain.com with your bastion hostname, and 172.20.0.0/16 with your VPC CIDR:
sshuttle -r ubuntu@bastion.domain.com 172.20.0.0/16
You can now access your private Kubernetes API using the internal API hostname in the KubeConfig, and SSH directly to Kubernetes nodes without any proxy configuration defined in your ssh_config file.
Press CTRL-c to kill sshuttle when you are done with the proxy. There are many useful sshuttle command-line options, such as running in the background, and specifying the CIDR to redirect in a file.
See the file example-usage for an example of how to use this module. Below are the available module inputs:
The following requirements are needed by this module:
The following providers are used by this module:
No modules.
The following resources are used by this module:
- aws_autoscaling_group.bastion (resource)
- aws_cloudwatch_log_group.bastion (resource)
- aws_iam_instance_profile.bastion (resource)
- aws_iam_role.bastion_role (resource)
- aws_iam_role_policy.bastion_logging (resource)
- aws_iam_role_policy.bastion_route53 (resource)
- aws_iam_role_policy.bastion_s3 (resource)
- aws_key_pair.bastion (resource)
- aws_launch_template.bastion (resource)
- aws_s3_bucket_object.additional-external-users-script (resource)
- aws_security_group.bastion_ssh (resource)
- aws_security_group_rule.bastion_egress (resource)
- aws_security_group_rule.bastion_ssh (resource)
- aws_ami.ubuntu (data source)
- aws_s3_bucket.infrastructure_bucket (data source)
- template_file.additional_external_user (data source)
- template_file.additional_user (data source)
- template_file.bastion_user_data (data source)
The following input variables are required:
Description: An S3 bucket to store data that should persist on the bastion when it is recycled by the Auto Scaling Group, such as SSH host keys. This can be set in the environment via TF_VAR_infrastructure_bucket
Type: any
Description: ID of the ROute53 zone for the bastion to add its host record.
Type: any
Description: An email address where unattended upgrade errors should be emailed. THis sets the option in /etc/apt/apt.conf.d/50unattended-upgrades
Type: any
Description: The VPC ID where the bastion and its security group will be created. This must match subnet IDs specified in the vpc_subnet_ids input.
Type: any
Description: A list of subnet IDs where the Auto Scaling Group can place the bastion.
Type: list
The following input variables are optional (have default values):
Description: Additional users to be created on the bastion. Works the same as additional_users, but adds users via a separate systemd unit file. Specify users as a list of maps. See an example in the example-usage file. Required map keys are login (user name) and authorized_keys. Optional map keys are gecos (full name), supplemental_groups (comma-separated), and shell. The authorized_keys will be output to ~/.ssh/authorized_keys using printf - multiple keys can be specified by including \n in the string.
Type: list
Default: []
Description: Content to be appended to UserData, which is run the first time the bastion EC2 boots.
Type: string
Default: ""
Description: Additional users to be created on the bastion. Specify users as a list of maps. See an example in the example-usage file. Required map keys are login (user name) and authorized_keys. Optional map keys are gecos (full name), supplemental_groups (comma-separated), and shell. The authorized_keys will be output to ~/.ssh/authorized_keys using printf - multiple keys can be specified by including \n in the string.
Type: list
Default: []
Description: The filter path for the AMI.
Type: string
Default: "ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*"
Description: The ID of the AMI's owner in AWS. The default is Canonical.
Type: string
Default: "099720109477"
Description: The ID of the AMI's owner in AWS GovCloud. This value is used automatically if the module's arn_prefix input variable is anything other than arn:aws. The default is Canonical.
Type: string
Default: "513442679011"
Description: The prefix to use for AWS ARNs.
Type: string
Default: "arn:aws"
Description: The name of the bastion EC2 instance, DNS hostname, CloudWatch Log Group, and the name prefix for other related resources.
Type: string
Default: "ro-bastion"
Description: Custom image ID. Use if you prefer a specific image to a standard Ubuntu image.
Type: string
Default: ""
Description: If true, encrypt the root ebs volume of the bastion
Type: bool
Default: true
Description: Extra tags for the bastion autoscaling group
Type: list
Default: []
Description: The key; sub-directory in $infrastructure_bucket where the bastion will be allowed to read and write. Do not specify a trailing slash. This allows sharing an S3 bucket among multiple invocations of this module.
Type: string
Default: "bastion"
Description: The EC2 instance type of the bastion.
Type: string
Default: "t3.micro"
Description: The number of days to retain logs in the CloudWatch Log Group.
Type: string
Default: "60"
Description: Whether to remove root access from the ubuntu user. Set this to yes|true|1 to remove root access, or anything else to retain it.
Type: string
Default: "true"
Description: The root volume type for the bastion instance
Type: string
Default: "gp3"
Description: A list of CIDRs allowed to SSH to the bastion. Override the module default by specifying an empty list, []
Type: list(string)
Default:
[
"0.0.0.0/0"
]Description: The content of an existing SSH public key file, that will be used to create an AWS SSH Key Pair. Yes, this input has an unfortunate name.
Type: string
Default: ""
Description: Additional configuration lines to add to /etc/apt/apt.conf.d/50unattended-upgrades
Type: string
Default: ""
Description: The time that the bastion should reboot, when necessary, after an an unattended upgrade. This sets the option in /etc/apt/apt.conf.d/50unattended-upgrades
Type: string
Default: "21:30"
The following outputs are exported:
Description: The ARN of the autoscaling group
Description: The ID of the autoscaling group
Description: The ID of the bastion security group
We are happy to share this internal module with the community. We appreciate suggestions for improvement, and recommend starting by opening an issue. Please see contributing.md for details.
The design document describes the goals and vision for this project.