Skip to content

Getting Started

Jump to bottom
ChrisDowning edited this page Jul 25, 2020 · 8 revisions

Pre-requisites

The content of this tutorial assumes you are familiar with the following:

  • Basic Linux system administration commands
  • Using SSH to access remote servers, including creating/adding keys
  • Setting environment variables

About AWS Account Access Permissions

You must have access to an AWS account to follow this tutorial.

Using a temporary account

As part of the event, we plan to make a small number of "burner" AWS accounts available for attendees to experiment with. These accounts do not require any billing setup, and can be accessed by entering the provided access key here. Temporary accounts will be automatically deleted after the event is concluded. Reach out to the organisers via Slack if you wish to be provided with a temporary account.

Using a standard account

Access should be via an IAM user login; using the root account credentials is not recommended. If you are working in a temporary or sandbox account, the "AdministratorAccess" IAM role will allow you to perform all of the required changes; this role will need to be applied to your user account, or can be attached to an EC2 instance. If you wish to use the AWS CLI with authentication as an IAM user, you will need to provide your API keys when configuring the CLI. If instead you wish to use a role attached to an EC2 instance, you will automatically have the required permissions once logged in to that instance, without the need for API keys.

If you are working within an organisationally-managed AWS account and do not have permissions to modify IAM users/roles, contact your account admin. They will be able to assist by either provisioning a role to your user account with appropriate (though likely more limited) permissions, or providing access to an EC2 instance which has the required role. Provisioning a static HPC cluster requires the ability to create EC2 instances and networking components, while deploying a dynamically scaling cluster additionally requires the creation of custom IAM policies and roles, which in turn allow instances to be created and terminated automatically.

All steps documented here can be completed without exceeding default AWS account quotas. Wherever possible, we suggest using t2.micro or t3.micro instances for testing; while these instances are not appropriate for real HPC workloads, they are low-cost and for new accounts can be used for a limited number of instance-hours per month for under the free tier. Keep in mind that you are responsible for ensuring the resources created within your account are properly deleted after use in order to avoid unexpected charges.

Using an EC2 instance as a bastion/entry-point

Using a bastion host as the entrypoint for any SSH connections into an AWS account is a simple way to reduce security risks. The bastion host instance should be configured to allow inbound connections only on port 22, and if possible only from a fixed IP address or IP range.

The process to create an EC2 instance via the AWS console is described here and the various methods of connecting to the instance are documented here.

For users wishing to interact with AWS using the CLI from their local device, API keys will need to be created for their IAM user account. These keys should then be set either within the AWS CLI config file or the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables of the user. For more details, see the documentation here.

If you choose to use an instance profile rather than user-owned API keys, you will need to create a profile based on the desired IAM role - the process for completing these steps is described here.

Key points to be aware of:

  • The default username on an EC2 instance depends on the AMI - for marketplace images, the most common choices are likely to be ec2-user (for Amazon Linux and Amazon Linux 2), centos for base CentOS images, or ubuntu for Ubuntu images. Using any of these images as the OS for a simple SSH access point is fine, however CentOS/SUSE should be used for any instances where the OpenHPC software stack is to be installed.
  • ...

Using the Cloud9 IDE as a bastion/entry-point

AWS Cloud9 is a browser-based IDE and terminal which can be launched and accessed directly from the AWS console. Using Cloud9 removes the need for configuring SSH access to bastion host from a local device. Additionally, Cloud9 will automatically power down its underlying EC2 instance after a (configurable) period of idle time.

Guidance on using Cloud9 can be found here - in most cases, "Individual User Setup" will be the right approach. Note that Cloud9 supports only Amazon Linux and Ubuntu operating systems; either of these is OK for the purposes of providing a reliable CLI and access point to the account.

To launch a simple Cloud9 environment in either an EventEngine account or your own account, you can deploy the following CloudFormation template (either from the CloudFormation console, or using the AWS CLI):

Resources:
  IDE:
    Type: AWS::Cloud9::EnvironmentEC2
    Properties:
      Description: Cloud9 IDE
      AutomaticStopTimeMinutes: 30
      InstanceType: t3.micro
      Name: "OpenHPC Cloud9"
Clone this wiki locally