docs/user/aws/install_upi: Explain DNS-zone lookup #2420

wking · 2019-09-27T04:14:27Z

We shouldn't assume folks will have a private zone they can dedicate to the sole use of the new cluster. This pull-request talks users through adjusting their DNS configuration to consume an existing private zone with arbitrary identification.

CC @ewolinetz, @kalexand-rh

wking · 2019-09-27T05:33:10Z

/test e2e-aws-upi

wking · 2019-09-27T13:31:35Z

UPI failed with:

level=fatal msg="failed to initialize the cluster: Some cluster operators are still updating: authentication, console"

Which makes sense for "ingress couldn't find the internal zone", because we haven't patched the CI template to add the new Python.

abhinavdahiya · 2019-09-27T17:34:45Z

We shouldn't assume folks will have a private zone they can dedicate to the sole use of the new cluster.
the private zone is on the cluster domain which is pretty unique, and it is probably not too bad to have that for the cluster.

Also let's keep in mind these UPI templates are reference only we don't have to keep all the knobs..
a) we already tell people we need these records, so they can do it however they like.
b) keeping it as-is is easy for us to test.

so i would need this PR to make a stronger case of changing this example for enhancement as this is definitely not a bug fix.

wking · 2019-09-27T18:03:15Z

...templates are reference only we don't have to keep all the knobs...

I'm not adding a new knob to a template, I'm changing to a new hard-coded value that is more appropriate for UPI.

we already tell people we need these records, so they can do it however they like.

My initial motivation is supporting the UPI approach shuffle here, where we're actually trying to plug a cluster into preexisting infrastructure (instead of having the advantage of being able to call create ignition-configs first). I think supporting that flow in CI is important, and I think having destroy cluster not reap a prexisting private zone is important. This PR resolves both of those points (unless there is some further trouble with infra-name mismatches that I haven't tripped over yet).

This PR also lets us drop the previous Configure Router for UPI section which showed the DNS config but didn't suggest any user actions to take based on its output, and I think getting rid of that section is an improvement.

abhinavdahiya · 2019-09-27T18:13:36Z

where we're actually trying to plug a cluster into preexisting infrastructure (instead of having the advantage of being able to call create ignition-configs first).

why the pre-existing Private Zone in the templates themselves..
^^ which will be enhancement,.

I think supporting that flow in CI is important, and I think having destroy cluster not reap a pre-existing private zone is important.

^^ why? we can have our CI (or reference) use a per cluster private dns zone resource

abhinavdahiya · 2019-09-27T18:15:31Z

This PR also lets us drop the previous Configure Router for UPI section which showed the DNS config but didn't suggest any user actions to take based on its output, and I think getting rid of that section is an improvement.

https://storage.googleapis.com/origin-ci-test/pr-logs/pull/openshift_installer/2420/pull-ci-openshift-installer-master-e2e-aws-upi/11/artifacts/e2e-aws-upi/must-gather/cluster-scoped-resources/config.openshift.io/clusteroperators/authentication.yaml

the cluster operator fails because not the template needs to provide the DNS records compared to the ingress-operator.

So this is not improvement, rather changing behavior.

…i-e2e: Update AWS private Route 53 discovery We shouldn't assume folks will have a private zone they can dedicate to the sole use of the new cluster. This commit applies the manifest update suggested in [1], and uses 'sed' to update the CloudFormation template even for older installer branches that don't have the template change from [1]. [1]: openshift/installer#2420

abhinavdahiya · 2019-10-11T16:40:51Z

I am okay if we want to document how to let ingress controller control DNS records in a private zone of the user.

but changing the UPI template to use shared is not the correct path, as the UPI is one reference and it's fine if we choose to let ingress-operator control the private zone created for the cluster.

jstuever · 2019-10-11T20:44:11Z

IMHO we should document only two options... 1) create a new one via templates (and own it) or b) Alternatively, you can also remove tags completely and use the zone ID instead. I think all the fuss over shared creates unnecessary complexity.

wking · 2019-10-11T21:37:20Z

...Alternatively, you can also remove tags completely and use the zone ID instead...

Done in 4def0457b -> fbceb82b5.

...as the UPI is one reference and it's fine if we choose to let ingress-operator control the private zone created for the cluster...

This also makes sense to me. But I expect "I'm providing a private zone, here's it's ID" to be more common than "I'm providing a private zone, and I've conveniently tagged it with the owned tag you were expecting with your expected infrastructure name". Of course, we'll have to update our UPI templates in openshift/release to start providing that private zone, which I can do if/when this lands.

abhinavdahiya · 2019-10-11T21:52:39Z

The doc addition is fine.

This also makes sense to me. But I expect "I'm providing a private zone, here's it's ID" to be more common than "I'm providing a private zone, and I've conveniently tagged it with the owned tag you were expecting with your expected infrastructure name". Of course, we'll have to update our UPI templates in openshift/release to start providing that private zone, which I can do if/when this lands.

I am a hard No on increasing the complexity of our UPI reference and tests anymore on this specific case.

wking · 2019-10-11T22:53:57Z

I am a hard No on increasing the complexity of our UPI reference...

I still think "reference by ID" is simpler than "add the owned tag we expect from IPI". But I've pushed fbceb82b5 -> 77e7c2878 to punt on that and make this PR just the doc change.

abhinavdahiya · 2019-10-11T23:21:22Z

docs/user/aws/install_upi.md

+
+#### Identify the internal DNS zone
+
+If you want [the ingress operator][ingress-operator] to manage DNS records on your behalf, adjust the `privateZone` section in the DNS configuration to identify the zone it should use.


AFAIK ingress-operator adds the dns records to for clusterdomain only. Can it handle any other?

AFAIK ingress-operator adds the dns records to for clusterdomain only.

Ingress is filling DNSRecord requests. In most cases now this is just going to be the default-wildcard record, and I dropped an example YAML dump of that into the 14e0691 commit message for the curious ;). But folks pushing new DNSRecords will probably get them filled, because I don't think the ingress operator has (or should have) guards about which DNSRecords it should or should not fill. If you're authorized to push a DNSRecord where ingress is looking, you should get it filled.

But in case where the private dns zone id is for not-clusterdomain. then the DNSRecord created by default would fail right.

But in case where the private dns zone id is for not-clusterdomain. then the DNSRecord created by default would fail right.

Ah, probably. Although it depends on the zone domains. So you could have an example.com zone, and Ingress could create an a.example.com record even if your cluster was c.d.e.example.com (I think ;). But yeah, I expect it would fail to create example.edu. Does that matter? We aren't telling them to create DNSRecords here, we're just telling them how to feed zones to the ingress operator, which can have its own docs about what it can and cannot support.

We shouldn't assume folks will have a private zone they can dedicate to the sole use of the new cluster. This commit talks users through adjusting their DNS configuration to consume an existing zone with arbitrary identification. I'd like to drop the owned tag from 01_vpc.yaml, but that's been contentious [1]. I'm punting in this commit so we can get the consensus doc change landed. [1]: openshift#2420 (comment)

openshift-ci-robot · 2019-10-12T03:24:12Z

@wking: The following tests failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
ci/prow/e2e-aws-upi	4def0457b56c7cfe43da35f0f1482d02e782c86e	link	`/test e2e-aws-upi`
ci/prow/e2e-aws-disruptive	4def0457b56c7cfe43da35f0f1482d02e782c86e	link	`/test e2e-aws-disruptive`
ci/prow/e2e-aws-scaleup-rhel7	`639c38e`	link	`/test e2e-aws-scaleup-rhel7`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

jstuever

/lgtm

openshift-ci-robot · 2019-10-14T21:58:40Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jstuever, wking

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [wking]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

We shouldn't assume folks will have a private zone they can dedicate to the sole use of the new cluster. This commit talks users through adjusting their DNS configuration to consume an existing zone with arbitrary identification. I'd like to drop the owned tag from 01_vpc.yaml, but that's been contentious [1]. I'm punting in this commit so we can get the consensus doc change landed. [1]: openshift#2420 (comment)

openshift-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Sep 27, 2019

openshift-ci-robot requested review from jstuever and patrickdillon September 27, 2019 04:14

openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 27, 2019

wking force-pushed the aws-dns-shared-private-zone branch 2 times, most recently from 5087891 to 4def045 Compare September 27, 2019 04:20

wking mentioned this pull request Sep 27, 2019

ci-operator/templates/openshift/installer/cluster-launch-installer-upi-e2e: Update AWS private Route 53 discovery openshift/release#5208

Closed

wking force-pushed the aws-dns-shared-private-zone branch from 4def045 to fbceb82 Compare October 11, 2019 21:34

wking force-pushed the aws-dns-shared-private-zone branch from fbceb82 to 77e7c28 Compare October 11, 2019 22:52

wking changed the title ~~upi/aws/cloudformation/02_cluster_infra: Use a 'shared' private zone~~ docs/user/aws/install_upi: Explain internal-DNS-zone lookup Oct 11, 2019

abhinavdahiya reviewed Oct 11, 2019

View reviewed changes

wking changed the title ~~docs/user/aws/install_upi: Explain internal-DNS-zone lookup~~ docs/user/aws/install_upi: Explain DNS-zone lookup Oct 11, 2019

wking force-pushed the aws-dns-shared-private-zone branch from 77e7c28 to 639c38e Compare October 11, 2019 23:23

jstuever approved these changes Oct 14, 2019

View reviewed changes

openshift-ci-robot assigned jstuever Oct 14, 2019

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Oct 14, 2019

openshift-merge-robot merged commit f0b966c into openshift:master Oct 14, 2019

wking deleted the aws-dns-shared-private-zone branch October 17, 2019 17:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docs/user/aws/install_upi: Explain DNS-zone lookup #2420

docs/user/aws/install_upi: Explain DNS-zone lookup #2420

wking commented Sep 27, 2019

wking commented Sep 27, 2019

wking commented Sep 27, 2019 •

edited

Loading

abhinavdahiya commented Sep 27, 2019

wking commented Sep 27, 2019

abhinavdahiya commented Sep 27, 2019

abhinavdahiya commented Sep 27, 2019

abhinavdahiya commented Oct 11, 2019

jstuever commented Oct 11, 2019 •

edited

Loading

wking commented Oct 11, 2019

abhinavdahiya commented Oct 11, 2019

wking commented Oct 11, 2019

abhinavdahiya Oct 11, 2019

wking Oct 11, 2019 •

edited

Loading

abhinavdahiya Oct 11, 2019 •

edited

Loading

wking Oct 11, 2019

openshift-ci-robot commented Oct 12, 2019

jstuever left a comment

openshift-ci-robot commented Oct 14, 2019


		#### Identify the internal DNS zone

		If you want [the ingress operator][ingress-operator] to manage DNS records on your behalf, adjust the `privateZone` section in the DNS configuration to identify the zone it should use.

docs/user/aws/install_upi: Explain DNS-zone lookup #2420

docs/user/aws/install_upi: Explain DNS-zone lookup #2420

Conversation

wking commented Sep 27, 2019

wking commented Sep 27, 2019

wking commented Sep 27, 2019 • edited Loading

abhinavdahiya commented Sep 27, 2019

wking commented Sep 27, 2019

abhinavdahiya commented Sep 27, 2019

abhinavdahiya commented Sep 27, 2019

abhinavdahiya commented Oct 11, 2019

jstuever commented Oct 11, 2019 • edited Loading

wking commented Oct 11, 2019

abhinavdahiya commented Oct 11, 2019

wking commented Oct 11, 2019

abhinavdahiya Oct 11, 2019

Choose a reason for hiding this comment

wking Oct 11, 2019 • edited Loading

Choose a reason for hiding this comment

abhinavdahiya Oct 11, 2019 • edited Loading

Choose a reason for hiding this comment

wking Oct 11, 2019

Choose a reason for hiding this comment

openshift-ci-robot commented Oct 12, 2019

jstuever left a comment

Choose a reason for hiding this comment

openshift-ci-robot commented Oct 14, 2019

wking commented Sep 27, 2019 •

edited

Loading

jstuever commented Oct 11, 2019 •

edited

Loading

wking Oct 11, 2019 •

edited

Loading

abhinavdahiya Oct 11, 2019 •

edited

Loading