[CI] assume AWS role via GitHub OIDC for sccache S3

[paultiq]

Describe the enhancement requested

Summary

Several crossbow workflows use sccache w/ S3 for build caching. They are currently configured to use long-lived access key / secret key pairs stored as github secrets.

An alternative approach exists w/ temporary credentials: by registering GitHub's OIDC provider and assuming an AWS_ROLE_ARN that's been properly trusted.

Note: if the proposal is adopted (to standardize on ccache) in #50722, this is moot.
Note 2: I have this working on a fork (I use this setup elsewhere), and can share a draft PR if wanted.

References:

• AWS Well-architected Framework - Best Practices: SEC02-BP02 Use temporary credentials
• sccache documentation: S3 credentials
• Use IAM roles to connect GitHub Actions to actions in AWS

Current State

compose.yaml's x-sccache anchor passes AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY through to the build containers, inherited from the runner environment:

arrow/compose.yaml

Lines 74 to 75 in 18146fc

	AWS_ACCESS_KEY_ID:
	AWS_SECRET_ACCESS_KEY:

In GitHub Actions, these come from repository secrets: long-term IAM user credentials / secret-access key pairs:

arrow/dev/tasks/macros.jinja

Lines 292 to 293 in 18146fc

	"AWS_ACCESS_KEY_ID": '${{ secrets.AWS_ACCESS_KEY_ID }}',
	"AWS_SECRET_ACCESS_KEY": '${{ secrets.AWS_SECRET_ACCESS_KEY }}',

Proposal: AssumeRoleWithWebIdentity w/ AWS_ROLE_ARN

By assuming the AWS_ROLE_ARN, the sccache session relies on credentials that expire*, issued only to workflows in the trusted repository, and scoped to the sccache bucket.

This requires a few things:

• create a role in IAM with the bucket and object level permissions,
• trust the github oidc-provider,
• define AWS_ROLE_ARN and set id-token permission
• grant sts:AssumeRoleWithWebIdentity for the specific repositories or organization.
• pass AWS_SESSION_TOKEN
• add a aws-actions/configure-aws-credentials@v4 step to the workflow.

* default is 1 hour, configurable

Component(s)

Continuous Integration

[KHARSHAVARDHAN-eng]

@paultiq , i would like to work on this issue!!