Make the SSL certificate ARN configurable on an existing cluster

[RobertLucian]

Implementation

Can be done just by editing the API service's respective annotation: service.beta.kubernetes.io/aws-load-balancer-ssl-cert.
To make it simpler to implement, we could only allow this to be changed if the user has initially provided an ACM.

As for where this could fit, we could add another generic CLI command that would only allow changing the SSL ARN for now.
The command could be cortex cluster update <field-to-update> <value-to-update-it-to>.

Context

Requested by Oldřich Šafář from the community Slack.

Temporary workaround

Install kubectl, run kubectl edit service ingressgateway-apis -n istio-system, and update the service.beta.kubernetes.io/aws-load-balancer-ssl-cert annotation accordingly.

Notes

• Update the networking docs: the SSL certificate section of the custom domain guide can be moved into the HTTPS guide.

Relevant code

istio.yaml.j2

{% if config.get('ssl_certificate_arn', '') != '' %}
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "{{ config['ssl_certificate_arn'] }}"
{% endif %}

apis.yaml.j2

{% if config.get('ssl_certificate_arn', '') == '' %}
- port:
    number: 443
    name: https
    protocol: HTTPS
  hosts:
    - "*"
  tls:
    mode: SIMPLE
    serverCertificate: /etc/istio/customgateway-certs/tls.crt
    privateKey: /etc/istio/customgateway-certs/tls.key
{% else %}
- port:
    number: 443
    name: https
    protocol: HTTP
  hosts:
    - "*"
{% endif %}

cluster_config.go

if cc.SSLCertificateARN != nil {
	exists, err := awsClient.DoesCertificateExist(*cc.SSLCertificateARN)
	if err != nil {
		return errors.Wrap(err, SSLCertificateARNKey)
	}

	if !exists {
		return errors.Wrap(ErrorSSLCertificateARNNotFound(*cc.SSLCertificateARN, cc.Region), SSLCertificateARNKey)
	}
}