fix: Update module versions and example to latest to resolve errors

Author: bryantbiggs

README.md

@@ -5,6 +5,7 @@ Terraform module which creates AWS EMR resources.
 [![SWUbanner](https://raw.githubusercontent.com/vshymanskyy/StandWithUkraine/main/banner2-direct.svg)](https://github.com/vshymanskyy/StandWithUkraine/blob/main/docs/README.md)
 
 This module supports the creation of:
+
 - EMR clusters using instance fleets or instance groups deployed in public or private subnets
 - EMR Virtual clusters that run on Amazon EKS
 - EMR Serverless clusters
@@ -13,7 +14,8 @@ This module supports the creation of:
 - Security group for EMR `service` to support private clusters
 - IAM roles for autoscaling, EMR `service`, and EC2 instance profiles
 
-  :information_source: The appropriate resources have been tagged with `{ "for-use-with-amazon-emr-managed-policies" = true }` to support the use of the recommended IAM policy `"arn:aws:iam::aws:policy/service-role/AmazonEMRServicePolicy_v2"`. Users are required to tag the appropriate VPC resources (VPC and subnets) as needed. See [here](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-managed-iam-policies.html) for more details regarding v2 of managed EMR policies and their usage requirements.
+> [!IMPORTANT]
+> The appropriate resources have been tagged with `{ "for-use-with-amazon-emr-managed-policies" = true }` to support the use of the recommended IAM policy `"arn:aws:iam::aws:policy/service-role/AmazonEMRServicePolicy_v2"`. Users are required to tag the appropriate VPC resources (VPC and subnets) as needed. See [here](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-managed-iam-policies.html) for more details regarding v2 of managed EMR policies and their usage requirements.
 
 ## Usage
 

examples/private-cluster/README.md

@@ -43,9 +43,9 @@ Note that this example may create resources which will incur monetary charges on
 | <a name="module_emr_disabled"></a> [emr\_disabled](#module\_emr\_disabled) | ../.. | n/a |
 | <a name="module_emr_instance_fleet"></a> [emr\_instance\_fleet](#module\_emr\_instance\_fleet) | ../.. | n/a |
 | <a name="module_emr_instance_group"></a> [emr\_instance\_group](#module\_emr\_instance\_group) | ../.. | n/a |
-| <a name="module_s3_bucket"></a> [s3\_bucket](#module\_s3\_bucket) | terraform-aws-modules/s3-bucket/aws | ~> 4.0 |
-| <a name="module_vpc"></a> [vpc](#module\_vpc) | terraform-aws-modules/vpc/aws | ~> 5.0 |
-| <a name="module_vpc_endpoints"></a> [vpc\_endpoints](#module\_vpc\_endpoints) | terraform-aws-modules/vpc/aws//modules/vpc-endpoints | ~> 5.0 |
+| <a name="module_s3_bucket"></a> [s3\_bucket](#module\_s3\_bucket) | terraform-aws-modules/s3-bucket/aws | ~> 5.0 |
+| <a name="module_vpc"></a> [vpc](#module\_vpc) | terraform-aws-modules/vpc/aws | ~> 6.0 |
+| <a name="module_vpc_endpoints"></a> [vpc\_endpoints](#module\_vpc\_endpoints) | terraform-aws-modules/vpc/aws//modules/vpc-endpoints | ~> 6.0 |
 | <a name="module_vpc_endpoints_sg"></a> [vpc\_endpoints\_sg](#module\_vpc\_endpoints\_sg) | terraform-aws-modules/security-group/aws | ~> 5.0 |
 
 ## Resources
@@ -62,7 +62,6 @@ Note that this example may create resources which will incur monetary charges on
 | [aws_iam_policy_document.assume](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
 | [aws_iam_policy_document.autoscaling](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
 | [aws_partition.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) | data source |
-| [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source |
 
 ## Inputs
 

examples/private-cluster/main.tf

@@ -3,13 +3,9 @@ provider "aws" {
 }
 
 data "aws_availability_zones" "available" {}
-
 data "aws_partition" "current" {}
-
 data "aws_caller_identity" "current" {}
 
-data "aws_region" "current" {}
-
 locals {
   name   = replace(basename(path.cwd), "-cluster", "")
   region = "eu-west-1"
@@ -270,7 +266,7 @@ module "emr_disabled" {
 
 module "vpc" {
   source  = "terraform-aws-modules/vpc/aws"
-  version = "~> 5.0"
+  version = "~> 6.0"
 
   name = local.name
   cidr = local.vpc_cidr
@@ -294,7 +290,7 @@ module "vpc" {
 
 module "vpc_endpoints" {
   source  = "terraform-aws-modules/vpc/aws//modules/vpc-endpoints"
-  version = "~> 5.0"
+  version = "~> 6.0"
 
   vpc_id             = module.vpc.vpc_id
   security_group_ids = [module.vpc_endpoints_sg.security_group_id]
@@ -343,7 +339,7 @@ module "vpc_endpoints_sg" {
 
 module "s3_bucket" {
   source  = "terraform-aws-modules/s3-bucket/aws"
-  version = "~> 4.0"
+  version = "~> 5.0"
 
   bucket_prefix = "${local.name}-"
 
@@ -354,11 +350,6 @@ module "s3_bucket" {
   attach_deny_insecure_transport_policy = true
   attach_require_latest_tls_policy      = true
 
-  block_public_acls       = true
-  block_public_policy     = true
-  ignore_public_acls      = true
-  restrict_public_buckets = true
-
   server_side_encryption_configuration = {
     rule = {
       apply_server_side_encryption_by_default = {
@@ -427,7 +418,7 @@ data "aws_iam_policy_document" "autoscaling" {
     condition {
       test     = "ArnLike"
       variable = "aws:SourceArn"
-      values   = ["arn:aws:elasticmapreduce:${data.aws_region.current.name}:${data.aws_caller_identity.current.account_id}:*"]
+      values   = ["arn:aws:elasticmapreduce:${local.region}:${data.aws_caller_identity.current.account_id}:*"]
     }
   }
 }

examples/public-cluster/README.md

@@ -40,8 +40,8 @@ Note that this example may create resources which will incur monetary charges on
 |------|--------|---------|
 | <a name="module_emr_instance_fleet"></a> [emr\_instance\_fleet](#module\_emr\_instance\_fleet) | ../.. | n/a |
 | <a name="module_emr_instance_group"></a> [emr\_instance\_group](#module\_emr\_instance\_group) | ../.. | n/a |
-| <a name="module_s3_bucket"></a> [s3\_bucket](#module\_s3\_bucket) | terraform-aws-modules/s3-bucket/aws | ~> 4.0 |
-| <a name="module_vpc"></a> [vpc](#module\_vpc) | terraform-aws-modules/vpc/aws | ~> 5.0 |
+| <a name="module_s3_bucket"></a> [s3\_bucket](#module\_s3\_bucket) | terraform-aws-modules/s3-bucket/aws | ~> 5.0 |
+| <a name="module_vpc"></a> [vpc](#module\_vpc) | terraform-aws-modules/vpc/aws | ~> 6.0 |
 
 ## Resources
 

examples/public-cluster/main.tf

@@ -247,7 +247,7 @@ module "emr_instance_group" {
 
 module "vpc" {
   source  = "terraform-aws-modules/vpc/aws"
-  version = "~> 5.0"
+  version = "~> 6.0"
 
   name = local.name
   cidr = local.vpc_cidr
@@ -270,7 +270,7 @@ module "vpc" {
 
 module "s3_bucket" {
   source  = "terraform-aws-modules/s3-bucket/aws"
-  version = "~> 4.0"
+  version = "~> 5.0"
 
   bucket_prefix = "${local.name}-"
 
@@ -281,11 +281,6 @@ module "s3_bucket" {
   attach_deny_insecure_transport_policy = true
   attach_require_latest_tls_policy      = true
 
-  block_public_acls       = true
-  block_public_policy     = true
-  ignore_public_acls      = true
-  restrict_public_buckets = true
-
   server_side_encryption_configuration = {
     rule = {
       apply_server_side_encryption_by_default = {

examples/serverless-cluster/README.md

@@ -41,7 +41,7 @@ Note that this example may create resources which will incur monetary charges on
 | <a name="module_emr_serverless_disabled"></a> [emr\_serverless\_disabled](#module\_emr\_serverless\_disabled) | ../../modules/serverless | n/a |
 | <a name="module_emr_serverless_hive"></a> [emr\_serverless\_hive](#module\_emr\_serverless\_hive) | ../../modules/serverless | n/a |
 | <a name="module_emr_serverless_spark"></a> [emr\_serverless\_spark](#module\_emr\_serverless\_spark) | ../../modules/serverless | n/a |
-| <a name="module_vpc"></a> [vpc](#module\_vpc) | terraform-aws-modules/vpc/aws | ~> 5.0 |
+| <a name="module_vpc"></a> [vpc](#module\_vpc) | terraform-aws-modules/vpc/aws | ~> 6.0 |
 
 ## Resources
 

examples/serverless-cluster/main.tf

@@ -137,7 +137,7 @@ module "emr_serverless_disabled" {
 
 module "vpc" {
   source  = "terraform-aws-modules/vpc/aws"
-  version = "~> 5.0"
+  version = "~> 6.0"
 
   name = local.name
   cidr = local.vpc_cidr

examples/studio/README.md

@@ -38,9 +38,9 @@ $ terraform apply
 | <a name="module_emr_studio_disabled"></a> [emr\_studio\_disabled](#module\_emr\_studio\_disabled) | ../../modules/studio | n/a |
 | <a name="module_emr_studio_iam"></a> [emr\_studio\_iam](#module\_emr\_studio\_iam) | ../../modules/studio | n/a |
 | <a name="module_emr_studio_sso"></a> [emr\_studio\_sso](#module\_emr\_studio\_sso) | ../../modules/studio | n/a |
-| <a name="module_kms"></a> [kms](#module\_kms) | terraform-aws-modules/kms/aws | ~> 2.0 |
-| <a name="module_s3_bucket"></a> [s3\_bucket](#module\_s3\_bucket) | terraform-aws-modules/s3-bucket/aws | ~> 4.0 |
-| <a name="module_vpc"></a> [vpc](#module\_vpc) | terraform-aws-modules/vpc/aws | ~> 5.0 |
+| <a name="module_kms"></a> [kms](#module\_kms) | terraform-aws-modules/kms/aws | ~> 4.0 |
+| <a name="module_s3_bucket"></a> [s3\_bucket](#module\_s3\_bucket) | terraform-aws-modules/s3-bucket/aws | ~> 5.0 |
+| <a name="module_vpc"></a> [vpc](#module\_vpc) | terraform-aws-modules/vpc/aws | ~> 6.0 |
 
 ## Resources
 
@@ -49,7 +49,6 @@ $ terraform apply
 | [aws_availability_zones.available](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/availability_zones) | data source |
 | [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |
 | [aws_identitystore_group.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/identitystore_group) | data source |
-| [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source |
 | [aws_ssoadmin_instances.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssoadmin_instances) | data source |
 
 ## Inputs

examples/studio/main.tf

@@ -3,11 +3,8 @@ provider "aws" {
 }
 
 data "aws_availability_zones" "available" {}
-
 data "aws_caller_identity" "current" {}
 
-data "aws_region" "current" {}
-
 locals {
   name   = replace(basename(path.cwd), "-cluster", "")
   region = "eu-west-1"
@@ -174,7 +171,7 @@ module "emr_studio_disabled" {
 
 module "vpc" {
   source  = "terraform-aws-modules/vpc/aws"
-  version = "~> 5.0"
+  version = "~> 6.0"
 
   name = local.name
   cidr = local.vpc_cidr
@@ -192,7 +189,7 @@ module "vpc" {
 
 module "s3_bucket" {
   source  = "terraform-aws-modules/s3-bucket/aws"
-  version = "~> 4.0"
+  version = "~> 5.0"
 
   bucket_prefix = "${local.name}-"
 
@@ -221,7 +218,7 @@ module "s3_bucket" {
 
 module "kms" {
   source  = "terraform-aws-modules/kms/aws"
-  version = "~> 2.0"
+  version = "~> 4.0"
 
   deletion_window_in_days = 7
   description             = "KMS key for ${local.name}."
@@ -262,7 +259,7 @@ module "kms" {
         {
           test     = "StringEquals"
           variable = "kms:ViaService"
-          values   = ["s3.${data.aws_region.current.name}.amazonaws.com"]
+          values   = ["s3.${local.region}.amazonaws.com"]
         }
       ]
     }

examples/virtual-cluster/README.md

@@ -1,14 +1,10 @@
 # AWS EMR Virtual Cluster Example
 
-This example shows how to provision a serverless cluster (serverless data plane) using Fargate Profiles to support EMR on EKS virtual clusters.
-
-There are two Fargate profiles created:
-1. `kube-system` to support core Kubernetes components such as CoreDNS
-2. `emr-wildcard` which supports any namespaces that begin with `emr-*`; this allows for creating multiple virtual clusters without having to create additional Fargate profiles for each new cluster.
+This example shows how to provision a serverless cluster (serverless data plane) using EKS Auto Mode to support EMR on EKS virtual clusters.
 
 The resources created by the `virtual-cluster` module include:
+
 - Kubernetes namespace, role, and role binding; existing or externally created namespace and role can be utilized as well
-- IAM role for service account (IRSA) used by for job execution. Users can scope access to the appropriate S3 bucket and path via `s3_bucket_arns`, use for both accessing job data as well as writing out results. The bare minimum permissions have been provided for the job execution role; users can provide additional permissions by passing in additional policies to attach to the role via `iam_role_additional_policies`
 - CloudWatch log group for task execution logs. Log streams are created by the job itself and not via Terraform
 - EMR managed security group for the virtual cluster
 - EMR virtual cluster scoped to the namespace created/provided
@@ -18,9 +14,9 @@ The resources created by the `virtual-cluster` module include:
 To run this example you need to execute:
 
 ```bash
-$ terraform init
-$ terraform plan
-$ terraform apply
+terraform init
+terraform plan
+terraform apply
 ```
 
 Note that this example may create resources which will incur monetary charges on your AWS bill. Run `terraform destroy` when you no longer need these resources.
@@ -56,7 +52,6 @@ aws emr-containers list-virtual-clusters --region us-west-2 --states ARRESTED \
 |------|---------|
 | <a name="provider_aws"></a> [aws](#provider\_aws) | >= 5.83 |
 | <a name="provider_null"></a> [null](#provider\_null) | >= 3.0 |
-| <a name="provider_time"></a> [time](#provider\_time) | >= 0.7 |
 
 ## Modules
 
@@ -65,10 +60,10 @@ aws emr-containers list-virtual-clusters --region us-west-2 --states ARRESTED \
 | <a name="module_complete"></a> [complete](#module\_complete) | ../../modules/virtual-cluster | n/a |
 | <a name="module_default"></a> [default](#module\_default) | ../../modules/virtual-cluster | n/a |
 | <a name="module_disabled"></a> [disabled](#module\_disabled) | ../../modules/virtual-cluster | n/a |
-| <a name="module_eks"></a> [eks](#module\_eks) | terraform-aws-modules/eks/aws | ~> 19.13 |
-| <a name="module_s3_bucket"></a> [s3\_bucket](#module\_s3\_bucket) | terraform-aws-modules/s3-bucket/aws | ~> 4.0 |
-| <a name="module_vpc"></a> [vpc](#module\_vpc) | terraform-aws-modules/vpc/aws | ~> 5.0 |
-| <a name="module_vpc_endpoints"></a> [vpc\_endpoints](#module\_vpc\_endpoints) | terraform-aws-modules/vpc/aws//modules/vpc-endpoints | ~> 5.0 |
+| <a name="module_eks"></a> [eks](#module\_eks) | terraform-aws-modules/eks/aws | ~> 21.0 |
+| <a name="module_s3_bucket"></a> [s3\_bucket](#module\_s3\_bucket) | terraform-aws-modules/s3-bucket/aws | ~> 5.0 |
+| <a name="module_vpc"></a> [vpc](#module\_vpc) | terraform-aws-modules/vpc/aws | ~> 6.0 |
+| <a name="module_vpc_endpoints"></a> [vpc\_endpoints](#module\_vpc\_endpoints) | terraform-aws-modules/vpc/aws//modules/vpc-endpoints | ~> 6.0 |
 | <a name="module_vpc_endpoints_sg"></a> [vpc\_endpoints\_sg](#module\_vpc\_endpoints\_sg) | terraform-aws-modules/security-group/aws | ~> 5.0 |
 
 ## Resources
@@ -77,9 +72,7 @@ aws emr-containers list-virtual-clusters --region us-west-2 --states ARRESTED \
 |------|------|
 | [null_resource.s3_sync](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) | resource |
 | [null_resource.start_job_run](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) | resource |
-| [time_sleep.coredns](https://registry.terraform.io/providers/hashicorp/time/latest/docs/resources/sleep) | resource |
 | [aws_availability_zones.available](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/availability_zones) | data source |
-| [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |
 
 ## Inputs