Skip to content

Commit c5ee829

Browse files
authored
Merge pull request kubernetes#38551 from sftim/20221219_migrate_aws_specifics_from_service
Move AWS Service annotation details to reference page
2 parents aa8eabc + 5a98033 commit c5ee829

File tree

2 files changed

+416
-263
lines changed
  • content/en/docs

2 files changed

+416
-263
lines changed

content/en/docs/concepts/services-networking/service.md

-263
Original file line numberDiff line numberDiff line change
@@ -787,269 +787,6 @@ metadata:
787787
{{% /tab %}}
788788
{{< /tabs >}}
789789

790-
#### TLS support on AWS {#ssl-support-on-aws}
791-
792-
For partial TLS / SSL support on clusters running on AWS, you can add three
793-
annotations to a `LoadBalancer` service:
794-
795-
```yaml
796-
metadata:
797-
name: my-service
798-
annotations:
799-
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:us-east-1:123456789012:certificate/12345678-1234-1234-1234-123456789012
800-
```
801-
802-
The first specifies the ARN of the certificate to use. It can be either a
803-
certificate from a third party issuer that was uploaded to IAM or one created
804-
within AWS Certificate Manager.
805-
806-
```yaml
807-
metadata:
808-
name: my-service
809-
annotations:
810-
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: (https|http|ssl|tcp)
811-
```
812-
813-
The second annotation specifies which protocol a Pod speaks. For HTTPS and
814-
SSL, the ELB expects the Pod to authenticate itself over the encrypted
815-
connection, using a certificate.
816-
817-
HTTP and HTTPS selects layer 7 proxying: the ELB terminates
818-
the connection with the user, parses headers, and injects the `X-Forwarded-For`
819-
header with the user's IP address (Pods only see the IP address of the
820-
ELB at the other end of its connection) when forwarding requests.
821-
822-
TCP and SSL selects layer 4 proxying: the ELB forwards traffic without
823-
modifying the headers.
824-
825-
In a mixed-use environment where some ports are secured and others are left unencrypted,
826-
you can use the following annotations:
827-
828-
```yaml
829-
metadata:
830-
name: my-service
831-
annotations:
832-
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
833-
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443,8443"
834-
```
835-
836-
In the above example, if the Service contained three ports, `80`, `443`, and
837-
`8443`, then `443` and `8443` would use the SSL certificate, but `80` would be proxied HTTP.
838-
839-
From Kubernetes v1.9 onwards you can use
840-
[predefined AWS SSL policies](https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-security-policy-table.html)
841-
with HTTPS or SSL listeners for your Services.
842-
To see which policies are available for use, you can use the `aws` command line tool:
843-
844-
```bash
845-
aws elb describe-load-balancer-policies --query 'PolicyDescriptions[].PolicyName'
846-
```
847-
848-
You can then specify any one of those policies using the
849-
"`service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy`"
850-
annotation; for example:
851-
852-
```yaml
853-
metadata:
854-
name: my-service
855-
annotations:
856-
service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy: "ELBSecurityPolicy-TLS-1-2-2017-01"
857-
```
858-
859-
#### PROXY protocol support on AWS
860-
861-
To enable [PROXY protocol](https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt)
862-
support for clusters running on AWS, you can use the following service
863-
annotation:
864-
865-
```yaml
866-
metadata:
867-
name: my-service
868-
annotations:
869-
service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"
870-
```
871-
872-
Since version 1.3.0, the use of this annotation applies to all ports proxied by the ELB
873-
and cannot be configured otherwise.
874-
875-
#### ELB Access Logs on AWS
876-
877-
There are several annotations to manage access logs for ELB Services on AWS.
878-
879-
The annotation `service.beta.kubernetes.io/aws-load-balancer-access-log-enabled`
880-
controls whether access logs are enabled.
881-
882-
The annotation `service.beta.kubernetes.io/aws-load-balancer-access-log-emit-interval`
883-
controls the interval in minutes for publishing the access logs. You can specify
884-
an interval of either 5 or 60 minutes.
885-
886-
The annotation `service.beta.kubernetes.io/aws-load-balancer-access-log-s3-bucket-name`
887-
controls the name of the Amazon S3 bucket where load balancer access logs are
888-
stored.
889-
890-
The annotation `service.beta.kubernetes.io/aws-load-balancer-access-log-s3-bucket-prefix`
891-
specifies the logical hierarchy you created for your Amazon S3 bucket.
892-
893-
```yaml
894-
metadata:
895-
name: my-service
896-
annotations:
897-
# Specifies whether access logs are enabled for the load balancer
898-
service.beta.kubernetes.io/aws-load-balancer-access-log-enabled: "true"
899-
900-
# The interval for publishing the access logs. You can specify an interval of either 5 or 60 (minutes).
901-
service.beta.kubernetes.io/aws-load-balancer-access-log-emit-interval: "60"
902-
903-
# The name of the Amazon S3 bucket where the access logs are stored
904-
service.beta.kubernetes.io/aws-load-balancer-access-log-s3-bucket-name: "my-bucket"
905-
906-
# The logical hierarchy you created for your Amazon S3 bucket, for example `my-bucket-prefix/prod`
907-
service.beta.kubernetes.io/aws-load-balancer-access-log-s3-bucket-prefix: "my-bucket-prefix/prod"
908-
```
909-
910-
#### Connection Draining on AWS
911-
912-
Connection draining for Classic ELBs can be managed with the annotation
913-
`service.beta.kubernetes.io/aws-load-balancer-connection-draining-enabled` set
914-
to the value of `"true"`. The annotation
915-
`service.beta.kubernetes.io/aws-load-balancer-connection-draining-timeout` can
916-
also be used to set maximum time, in seconds, to keep the existing connections open before
917-
deregistering the instances.
918-
919-
```yaml
920-
metadata:
921-
name: my-service
922-
annotations:
923-
service.beta.kubernetes.io/aws-load-balancer-connection-draining-enabled: "true"
924-
service.beta.kubernetes.io/aws-load-balancer-connection-draining-timeout: "60"
925-
```
926-
927-
#### Other ELB annotations
928-
929-
There are other annotations to manage Classic Elastic Load Balancers that are described below.
930-
931-
```yaml
932-
metadata:
933-
name: my-service
934-
annotations:
935-
# The time, in seconds, that the connection is allowed to be idle (no data has been sent
936-
# over the connection) before it is closed by the load balancer
937-
service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "60"
938-
939-
# Specifies whether cross-zone load balancing is enabled for the load balancer
940-
service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true"
941-
942-
# A comma-separated list of key-value pairs which will be recorded as
943-
# additional tags in the ELB.
944-
service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags: "environment=prod,owner=devops"
945-
946-
# The number of successive successful health checks required for a backend to
947-
# be considered healthy for traffic. Defaults to 2, must be between 2 and 10
948-
service.beta.kubernetes.io/aws-load-balancer-healthcheck-healthy-threshold: ""
949-
950-
# The number of unsuccessful health checks required for a backend to be
951-
# considered unhealthy for traffic. Defaults to 6, must be between 2 and 10
952-
service.beta.kubernetes.io/aws-load-balancer-healthcheck-unhealthy-threshold: "3"
953-
954-
# The approximate interval, in seconds, between health checks of an
955-
# individual instance. Defaults to 10, must be between 5 and 300
956-
service.beta.kubernetes.io/aws-load-balancer-healthcheck-interval: "20"
957-
958-
# The amount of time, in seconds, during which no response means a failed
959-
# health check. This value must be less than the service.beta.kubernetes.io/aws-load-balancer-healthcheck-interval
960-
# value. Defaults to 5, must be between 2 and 60
961-
service.beta.kubernetes.io/aws-load-balancer-healthcheck-timeout: "5"
962-
963-
# A list of existing security groups to be configured on the ELB created. Unlike the annotation
964-
# service.beta.kubernetes.io/aws-load-balancer-extra-security-groups, this replaces all other
965-
# security groups previously assigned to the ELB and also overrides the creation
966-
# of a uniquely generated security group for this ELB.
967-
# The first security group ID on this list is used as a source to permit incoming traffic to
968-
# target worker nodes (service traffic and health checks).
969-
# If multiple ELBs are configured with the same security group ID, only a single permit line
970-
# will be added to the worker node security groups, that means if you delete any
971-
# of those ELBs it will remove the single permit line and block access for all ELBs that shared the same security group ID.
972-
# This can cause a cross-service outage if not used properly
973-
service.beta.kubernetes.io/aws-load-balancer-security-groups: "sg-53fae93f"
974-
975-
# A list of additional security groups to be added to the created ELB, this leaves the uniquely
976-
# generated security group in place, this ensures that every ELB
977-
# has a unique security group ID and a matching permit line to allow traffic to the target worker nodes
978-
# (service traffic and health checks).
979-
# Security groups defined here can be shared between services.
980-
service.beta.kubernetes.io/aws-load-balancer-extra-security-groups: "sg-53fae93f,sg-42efd82e"
981-
982-
# A comma separated list of key-value pairs which are used
983-
# to select the target nodes for the load balancer
984-
service.beta.kubernetes.io/aws-load-balancer-target-node-labels: "ingress-gw,gw-name=public-api"
985-
```
986-
987-
#### Network Load Balancer support on AWS {#aws-nlb-support}
988-
989-
{{< feature-state for_k8s_version="v1.15" state="beta" >}}
990-
991-
To use a Network Load Balancer on AWS, use the annotation `service.beta.kubernetes.io/aws-load-balancer-type` with the value set to `nlb`.
992-
993-
```yaml
994-
metadata:
995-
name: my-service
996-
annotations:
997-
service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
998-
```
999-
1000-
{{< note >}}
1001-
NLB only works with certain instance classes; see the
1002-
[AWS documentation](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/target-group-register-targets.html#register-deregister-targets)
1003-
on Elastic Load Balancing for a list of supported instance types.
1004-
{{< /note >}}
1005-
1006-
Unlike Classic Elastic Load Balancers, Network Load Balancers (NLBs) forward the
1007-
client's IP address through to the node. If a Service's `.spec.externalTrafficPolicy`
1008-
is set to `Cluster`, the client's IP address is not propagated to the end
1009-
Pods.
1010-
1011-
By setting `.spec.externalTrafficPolicy` to `Local`, the client IP addresses is
1012-
propagated to the end Pods, but this could result in uneven distribution of
1013-
traffic. Nodes without any Pods for a particular LoadBalancer Service will fail
1014-
the NLB Target Group's health check on the auto-assigned
1015-
`.spec.healthCheckNodePort` and not receive any traffic.
1016-
1017-
In order to achieve even traffic, either use a DaemonSet or specify a
1018-
[pod anti-affinity](/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity)
1019-
to not locate on the same node.
1020-
1021-
You can also use NLB Services with the [internal load balancer](/docs/concepts/services-networking/service/#internal-load-balancer)
1022-
annotation.
1023-
1024-
In order for client traffic to reach instances behind an NLB, the Node security
1025-
groups are modified with the following IP rules:
1026-
1027-
| Rule | Protocol | Port(s) | IpRange(s) | IpRange Description |
1028-
|------|----------|---------|------------|---------------------|
1029-
| Health Check | TCP | NodePort(s) (`.spec.healthCheckNodePort` for `.spec.externalTrafficPolicy = Local`) | Subnet CIDR | kubernetes.io/rule/nlb/health=\<loadBalancerName\> |
1030-
| Client Traffic | TCP | NodePort(s) | `.spec.loadBalancerSourceRanges` (defaults to `0.0.0.0/0`) | kubernetes.io/rule/nlb/client=\<loadBalancerName\> |
1031-
| MTU Discovery | ICMP | 3,4 | `.spec.loadBalancerSourceRanges` (defaults to `0.0.0.0/0`) | kubernetes.io/rule/nlb/mtu=\<loadBalancerName\> |
1032-
1033-
In order to limit which client IP's can access the Network Load Balancer,
1034-
specify `loadBalancerSourceRanges`.
1035-
1036-
```yaml
1037-
spec:
1038-
loadBalancerSourceRanges:
1039-
- "143.231.0.0/16"
1040-
```
1041-
1042-
{{< note >}}
1043-
If `.spec.loadBalancerSourceRanges` is not set, Kubernetes
1044-
allows traffic from `0.0.0.0/0` to the Node Security Group(s). If nodes have
1045-
public IP addresses, be aware that non-NLB traffic can also reach all instances
1046-
in those modified security groups.
1047-
1048-
{{< /note >}}
1049-
1050-
Further documentation on annotations for Elastic IPs and other common use-cases may be found
1051-
in the [AWS Load Balancer Controller documentation](https://kubernetes-sigs.github.io/aws-load-balancer-controller/latest/guide/service/annotations/).
1052-
1053790
### `type: ExternalName` {#externalname}
1054791

1055792

0 commit comments

Comments
 (0)