Skip to content

s3 uploader does not use credentials stored in ~/.aws/credentials #1069

@troyfontaine

Description

@troyfontaine

We're receiving the following error with the artifact upload functionality in the agent on macOS:

Failed to upload artifacts: Error creating uploader: Could not find a valid authentication strategy to connect to S3. Try setting BUILDKITE_S3_ACCESS_KEY and BUILDKITE_S3_SECRET_KEY

Reviewing the source code for the agent, it looks like the agent is dependent on the credentials being stored in environment variables and not checking if they're set in ~/.aws/credentials.

agent/agent/s3.go

Lines 21 to 43 in 0c29289

func (e *credentialsProvider) Retrieve() (creds credentials.Value, err error) {
e.retrieved = false
creds.AccessKeyID = os.Getenv("BUILDKITE_S3_ACCESS_KEY_ID")
if creds.AccessKeyID == "" {
creds.AccessKeyID = os.Getenv("BUILDKITE_S3_ACCESS_KEY")
}
creds.SecretAccessKey = os.Getenv("BUILDKITE_S3_SECRET_ACCESS_KEY")
if creds.SecretAccessKey == "" {
creds.SecretAccessKey = os.Getenv("BUILDKITE_S3_SECRET_KEY")
}
if creds.AccessKeyID == "" {
err = errors.New("BUILDKITE_S3_ACCESS_KEY_ID or BUILDKITE_S3_ACCESS_KEY not found in environment")
}
if creds.SecretAccessKey == "" {
err = errors.New("BUILDKITE_S3_SECRET_ACCESS_KEY or BUILDKITE_S3_SECRET_KEY not found in environment")
}
e.retrieved = true
return
}

The AWS SDK documentation states that you can pull in the values from the credentials file as follows:

sess, err := session.NewSession(&aws.Config{
    Region: aws.String("us-west-2")},
)

// Setup the S3 Upload Manager. Also see the SDK doc for the Upload Manager
// for more information on configuring part size, and concurrency.
//
// http://docs.aws.amazon.com/sdk-for-go/api/service/s3/s3manager/#NewUploader
uploader := s3manager.NewUploader(sess)

Perhaps a check to see if the default credentials file exists then use it for handling credentials, then fall back to the environment variables? Thanks!

Metadata

Metadata

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions