Unable to use "security_groups" in "node_security_group_additional_rules"

[tchwpkgorg]

Description

Terraform is not able to add a security group rule if I use "security_groups" in "node_security_group_additional_rules". It works fine if I use "cidr_blocks". I'm able to use and add "security_groups" with just the aws provider, without the terraform-aws-modules/eks/aws module.

apply fails, because "security_groups" is not even present in the plan, despite being present in the tf file.

Versions

• Module version [Required]:

Module version 20.33.1.

• Terraform version:
  Terraform v1.10.5
  on linux_amd64

• Provider version(s):
  Terraform v1.10.5
  on linux_amd64

• provider registry.terraform.io/hashicorp/aws v5.87.0
• provider registry.terraform.io/hashicorp/cloudinit v2.3.5
• provider registry.terraform.io/hashicorp/kubernetes v2.34.0
• provider registry.terraform.io/hashicorp/null v3.2.3
• provider registry.terraform.io/hashicorp/time v0.12.1
• provider registry.terraform.io/hashicorp/tls v4.0.6

Reproduction Code [Required]

Sample code:

  node_security_group_additional_rules = merge(

      ingress_vpc_to_cluster_port_6032 = {
        description = "Allow hosts in VPC to connect to PODs port 6032"
        protocol    = "tcp"
        from_port   = 6032
        to_port     = 6032
        type        = "ingress"
#        cidr_blocks = [data.aws_vpc.vpc.cidr_block]
        security_groups = [data.aws_security_group.proxysql.id]
      },
(...)
)

Steps to reproduce the behavior:

Expected behavior

Security group rule is added.

Actual behavior

"apply" exits with an error after 5 minutes.

Terminal Output Screenshot(s)

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create
 <= read (data resources)

Terraform will perform the following actions:

  # data.aws_eks_cluster.eks will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_eks_cluster" "eks" {
      + access_config             = (known after apply)
      + arn                       = (known after apply)
      + certificate_authority     = (known after apply)
      + cluster_id                = (known after apply)
      + compute_config            = (known after apply)
      + created_at                = (known after apply)
      + enabled_cluster_log_types = (known after apply)
      + endpoint                  = (known after apply)
      + id                        = (known after apply)
      + identity                  = (known after apply)
      + kubernetes_network_config = (known after apply)
      + name                      = "kt-staging-eks-cluster"
      + outpost_config            = (known after apply)
      + platform_version          = (known after apply)
      + remote_network_config     = (known after apply)
      + role_arn                  = (known after apply)
      + status                    = (known after apply)
      + storage_config            = (known after apply)
      + tags                      = (known after apply)
      + upgrade_policy            = (known after apply)
      + version                   = (known after apply)
      + vpc_config                = (known after apply)
      + zonal_shift_config        = (known after apply)
    }

  # module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"] will be created
  + resource "aws_security_group_rule" "node" {
      + description              = "Allow hosts in VPC to connect to PODs port 6032"
      + from_port                = 6032
      + id                       = (known after apply)
      + prefix_list_ids          = []
      + protocol                 = "tcp"
      + security_group_id        = "sg-aaabbbcccdddeee"
      + security_group_rule_id   = (known after apply)
      + self                     = false
      + source_security_group_id = (known after apply)
      + to_port                  = 6032
      + type                     = "ingress"
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Creating...
data.aws_eks_cluster.eks: Reading...
data.aws_eks_cluster.eks: Read complete after 0s [id=kt-staging-eks-cluster]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [10s elapsed]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [20s elapsed]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [30s elapsed]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [40s elapsed]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [50s elapsed]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [1m0s elapsed]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [1m10s elapsed]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [1m20s elapsed]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [1m30s elapsed]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [1m40s elapsed]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [1m50s elapsed]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [2m0s elapsed]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [2m10s elapsed]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [2m20s elapsed]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [2m30s elapsed]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [2m40s elapsed]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [2m50s elapsed]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [3m0s elapsed]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [3m10s elapsed]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [3m20s elapsed]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [3m30s elapsed]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [3m40s elapsed]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [3m50s elapsed]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [4m0s elapsed]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [4m10s elapsed]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [4m20s elapsed]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [4m30s elapsed]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [4m40s elapsed]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [4m50s elapsed]
module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"]: Still creating... [5m0s elapsed]
╷
│ Error: waiting for Security Group (sg-aaabbbcccdddeee) Rule (sgrule-802361034) create: couldn't find resource
│ 
│   with module.eks_cluster.aws_security_group_rule.node["ingress_vpc_to_cluster_port_6032"],
│   on .terraform/modules/eks_cluster/node_groups.tf line 230, in resource "aws_security_group_rule" "node":
│  230: resource "aws_security_group_rule" "node" {
│ 

Additional context

Interestingly, when using aws provider version 5.68 - it does NOT error. It exits with a success, but the rule is not added on AWS side.

[github-actions]

This issue has been automatically marked as stale because it has been open 30 days
with no activity. Remove stale label or comment or this issue will be closed in 10 days

[tchwpkgorg]

This is still happening.

[bryantbiggs]

thats because it needs to be source_security_group_id

[tchwpkgorg]

That's returned by AWS API though?

 + source_security_group_id = (known after apply)
  + security_group_rule_id   = (known after apply)