You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Add an Apache Spark version 3.5 workgroup to the CI CloudFormation stack #1303
The CI stack defines only a PySpark engine version 3 Spark workgroup.
An existing workgroup cannot be upgraded to Apache Spark version 3.5, and replacing pyathena-spark would break the current Spark tests, so both workgroups must exist side by side until the Calculation API cursors are removed.
Proposed change
In cloudformation/github_actions_oidc.yaml:
Add a new AWS::Athena::WorkGroup with SelectedEngineVersion: "Apache Spark version 3.5", an execution role, no ResultConfiguration, and a small spark-defaults classification (for example spark.dynamicAllocation.maxExecutors) to bound CI cost.
Add or reuse an execution role whose policy covers the session APIs and GetSessionEndpoint, without the Calculation and notebook actions.
Optionally set MonitoringConfiguration for session logs.
Add the workgroup name as a parameter, pass it to the test workflow as an environment variable (for example AWS_ATHENA_SPARK35_WORKGROUP), and document it with the other .env variables.
Activity