[Federation] Setting up CoreDNS as DNS provider for Cluster Federation #2810

shashidharatd · 2017-03-14T18:51:09Z

Addresses and Fixes: #2197

cc @madhusudancs @kubernetes/sig-federation-misc

This change is

madhusudancs · 2017-03-24T22:16:55Z

Reviewed 2 of 2 files at r1.
Review status: all files reviewed at latest revision, 14 unresolved discussions.

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 28 at r1 (raw file):

To deploy CoreDNS we shall make use of helm charts. CoreDNS will be
deployed with Etcd as the backend and should be pre-installed. Etcd

etcd is usually spelled with a lower case e. Also, please link to their website - https://coreos.com/etcd

Same comments for all other occurrences.

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 33 at r1 (raw file):

```shell
$ helm install --name etcd-operator --namespace ns stable/etcd-operator

Can we use a better, more descriptive namespace?

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 41 at r1 (raw file):

After deployment succeeds, Etcd can be accessed with
"http://etcd-cluster:2379" endpoint within the host cluster.

Remove the quotes and just linkify the URL.

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 44 at r1 (raw file):

CoreDNS default configuration should be customized to suit federation
and shown below is the Values.yaml, which overrides the default

Link to the default Values.yaml file?

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 64 at r1 (raw file):

 - `isClusterService` specifies whether CoreDNS should be deployed as
cluster-service, which is default. You need to set it to false, so that
CoreDNS is deployed as kubernetes application service.

What's the difference between the two types? I am not sure I know the distinction.

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 70 at r1 (raw file):

cluster.
 - Disable `middleware.kubernetes`, which is enabled by default by
setting `middleware.kubernetes.enabled` to false.

Do CoreDNS users understand this configuration? I don't know much about CoreDNS and I can't say whether I want this or not.

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 72 at r1 (raw file):

setting `middleware.kubernetes.enabled` to false.
 - Enable `middleware.etcd` by setting `middleware.etcd.enabled` to
true.

Same as above.

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 89 at r1 (raw file):

Federation control plane can be deployed using `kubefed init`. CoreDNS
can be chosen as DNS provider by specifying two additional parameters.

It is probably worth mentioning that middleware.etcd.zones and --dns-zone-name flag to kubefed init should match.

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 98 at r1 (raw file):

```shell
[Global]
etcd-endpoints = http://etcd-cluster.ns:2379

I would make the host name part consistent. I understand why you are doing this. But I think it is worth making it consistent everywhere. Using etcd-cluster.ns everywhere isn't harmful right?

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 99 at r1 (raw file):

[Global]
etcd-endpoints = http://etcd-cluster.ns:2379
zones = example.com

Trailing dot for consistency as we did here - #2899 (review)

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 103 at r1 (raw file):

 - `etcd-endpoints` is the endpoint to access Etcd.
 - `zones` is the federation domain for which CoreDNS is authoritative.

and say "which is same as --dns-zone-name"

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 105 at r1 (raw file):

 - `zones` is the federation domain for which CoreDNS is authoritative.

## Setup CoreDNS server in nameserver resolv chain

resolv.conf chain?

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 109 at r1 (raw file):

Once the federation control plane is deployed and federated clusters
are joined to the federation, you need to add the CoreDNS server to
pod's nameserver resolv chain in all the federated clusters as this

resolv.conf chain?

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 115 at r1 (raw file):

```shell
        - --server=/example.com/<CoreDNS endpoint>

Is it not possible to use the kube-dns ConfigMap for this yet?

Comments from Reviewable

shashidharatd · 2017-03-25T01:42:17Z

@madhusudancs, Handled review comments. PTAL

Review status: 1 of 2 files reviewed at latest revision, 14 unresolved discussions.

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 28 at r1 (raw file):

Previously, madhusudancs (Madhusudan.C.S) wrote…

etcd is usually spelled with a lower case e. Also, please link to their website - https://coreos.com/etcd

Same comments for all other occurrences.

ok. Done

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 33 at r1 (raw file):

Previously, madhusudancs (Madhusudan.C.S) wrote…

Can we use a better, more descriptive namespace?

used my-namespace instead of ns

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 41 at r1 (raw file):

Previously, madhusudancs (Madhusudan.C.S) wrote…

Remove the quotes and just linkify the URL.

removed the quotes

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 44 at r1 (raw file):

Previously, madhusudancs (Madhusudan.C.S) wrote…

Link to the default Values.yaml file?

CoreDNS chart is not merged yet, so the link would not be stable if i link it to PR. On the other hand Values.yaml in helm charts is well known, so i think user should not have any difficulty finding out the default configuration given the chart.

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 64 at r1 (raw file):

Previously, madhusudancs (Madhusudan.C.S) wrote…

What's the difference between the two types? I am not sure I know the distinction.

CoreDNS chart can be deployed basically as 2 variants.

if isClusterService is true, then CoreDNS is deployed similar to kube-dns addon.
if isClusterService is false, then CoreDNS is deployed as any other k8s app.

they basically differ in labels and annotations.

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 70 at r1 (raw file):

Previously, madhusudancs (Madhusudan.C.S) wrote…

Do CoreDNS users understand this configuration? I don't know much about CoreDNS and I can't say whether I want this or not.

CoreDNS users do understand about the middleware. I had given the reference Values.yaml above which can be used to deploy CoreDNS for federation dns server. This explanation is just to let know the user of the configuration details and i believe whoever wants to tweak the configuration would know how to fit it to their needs.

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 72 at r1 (raw file):

Previously, madhusudancs (Madhusudan.C.S) wrote…

Same as above.

Same as previous reply

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 89 at r1 (raw file):

Previously, madhusudancs (Madhusudan.C.S) wrote…

It is probably worth mentioning that middleware.etcd.zones and --dns-zone-name flag to kubefed init should match.

Added a Note

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 98 at r1 (raw file):

Previously, madhusudancs (Madhusudan.C.S) wrote…

I would make the host name part consistent. I understand why you are doing this. But I think it is worth making it consistent everywhere. Using etcd-cluster.ns everywhere isn't harmful right?

Agree, used etcd-cluster.my-namespace everywhere

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 99 at r1 (raw file):

Previously, madhusudancs (Madhusudan.C.S) wrote…

Trailing dot for consistency as we did here - #2899 (review)

Done

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 103 at r1 (raw file):

Previously, madhusudancs (Madhusudan.C.S) wrote…

and say "which is same as --dns-zone-name"

Added.

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 105 at r1 (raw file):

Previously, madhusudancs (Madhusudan.C.S) wrote…

resolv.conf chain?

Changed. Done

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 109 at r1 (raw file):

Previously, madhusudancs (Madhusudan.C.S) wrote…

resolv.conf chain?

Changed. Done

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 115 at r1 (raw file):

Previously, madhusudancs (Madhusudan.C.S) wrote…

Is it not possible to use the kube-dns ConfigMap for this yet?

kube-dns ConfigMap in the current release cannot handle nameserver listening on any port other than 53. I did a PR to specify port for nameserver, but that would be included in subsequent releases.
If CoreDNS is deployed as NodePort service. than it will be accessible on some random port which cannot be configured using kube-dns ConfigMap method.

Comments from Reviewable

csbell · 2017-03-31T17:18:31Z

Madhu -- can you lgtm the changes if they look good to you?

shashidharatd · 2017-04-04T04:41:11Z

@madhusudancs, the CoreDNS helm chart PR is merged now. request a LGTM from you. is the fixes ok?

clenimar · 2017-04-05T14:39:20Z

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 99 at r2 (raw file):

[Global]
etcd-endpoints = http://etcd-cluster.my-namespace:2379
zones = example.com.

Does the user need to own this domain name? (In the light of kubernetes/kubernetes#43599)

Comments from Reviewable

madhusudancs · 2017-04-05T22:18:03Z

@shashidharatd a few minor comments.

Reviewed 1 of 1 files at r2.
Review status: all files reviewed at latest revision, 8 unresolved discussions, some commit checks failed.

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 33 at r1 (raw file):

Previously, shashidharatd (Shashidhara T D) wrote…

used my-namespace instead of ns

I would have preferred a descriptive one. Something like dns or coredns, but this is good enough.

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 41 at r1 (raw file):

Previously, shashidharatd (Shashidhara T D) wrote…

removed the quotes

Can you please linkify this so that people following the doc can just click the link? [http://etcd-cluster.my-namespace:2379](http://etcd-cluster.my-namespace:2379)

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 44 at r1 (raw file):

Previously, shashidharatd (Shashidhara T D) wrote…

CoreDNS chart is not merged yet, so the link would not be stable if i link it to PR. On the other hand Values.yaml in helm charts is well known, so i think user should not have any difficulty finding out the default configuration given the chart.

Ok, fair enough.

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 64 at r1 (raw file):

Previously, shashidharatd (Shashidhara T D) wrote…

CoreDNS chart can be deployed basically as 2 variants.

if isClusterService is true, then CoreDNS is deployed similar to kube-dns addon.

if isClusterService is false, then CoreDNS is deployed as any other k8s app.

they basically differ in labels and annotations.

Oh I see what you mean. This defines whether the deployment is a cluster component. Is that right? Could you please clarify this in the doc. It is not clear right now.

If that's the case, should users care about it? What are the pros and cons?

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 70 at r1 (raw file):

Previously, shashidharatd (Shashidhara T D) wrote…

CoreDNS users do understand about the middleware. I had given the reference Values.yaml above which can be used to deploy CoreDNS for federation dns server. This explanation is just to let know the user of the configuration details and i believe whoever wants to tweak the configuration would know how to fit it to their needs.

Ok, cool.

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 72 at r1 (raw file):

Previously, shashidharatd (Shashidhara T D) wrote…

Same as previous reply

Ack.

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 115 at r1 (raw file):

Previously, shashidharatd (Shashidhara T D) wrote…

kube-dns ConfigMap in the current release cannot handle nameserver listening on any port other than 53. I did a PR to specify port for nameserver, but that would be included in subsequent releases.
If CoreDNS is deployed as NodePort service. than it will be accessible on some random port which cannot be configured using kube-dns ConfigMap method.

I see. Do you think it is worth documenting both these techniques along with your explanation above?

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 99 at r2 (raw file):

Previously, clenimar (Clenimar Filemon) wrote…

Does the user need to own this domain name? (In the light of kubernetes/kubernetes#43599)

Not necessary if you are using CoreDNS as the DNS provider. However, it is the user's (read: cluster operator's) responsibility to

Appropriately configure cluster's kube-dns to forward all requests to example.com. to CoreDNS.
Make CoreDNS the authoritative server for example.com.
Ensure external clients reach CoreDNS for the DNS queries for example.com..

Comments from Reviewable

shashidharatd · 2017-04-06T05:33:04Z

@madhusudancs, have tried adressing your comments. PTAL

Review status: 0 of 2 files reviewed at latest revision, 6 unresolved discussions.

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 33 at r1 (raw file):

Previously, madhusudancs (Madhusudan.C.S) wrote…

I would have preferred a descriptive one. Something like dns or coredns, but this is good enough.

Using something like coredns might have confused, since the keyword is used for other purposes also.

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 41 at r1 (raw file):

Previously, madhusudancs (Madhusudan.C.S) wrote…

Can you please linkify this so that people following the doc can just click the link? [http://etcd-cluster.my-namespace:2379](http://etcd-cluster.my-namespace:2379)

oh, i misunderstood the comment. Fixed Now.

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 64 at r1 (raw file):

Previously, madhusudancs (Madhusudan.C.S) wrote…

Oh I see what you mean. This defines whether the deployment is a cluster component. Is that right? Could you please clarify this in the doc. It is not clear right now.

If that's the case, should users care about it? What are the pros and cons?

For federation use case. CoreDNS has to be deployed as normal k8s app and this is not default while installing helm chart. So i have given the overriding configuration to deploy CoreDNS as a normal k8s app with etcd as middleware.

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 115 at r1 (raw file):

Previously, madhusudancs (Madhusudan.C.S) wrote…

I see. Do you think it is worth documenting both these techniques along with your explanation above?

Once the kube-dns configuration related PR is merged and available to users. users need not do this manual configuration at all. we should remove this section in future. So i feel we should leave out those details which we do take care of.

Comments from Reviewable

madhusudancs · 2017-04-08T18:49:47Z

Reviewed 2 of 2 files at r3.
Review status: all files reviewed at latest revision, 4 unresolved discussions, some commit checks failed.

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 33 at r1 (raw file):

Previously, shashidharatd (Shashidhara T D) wrote…

Using something like coredns might have confused, since the keyword is used for other purposes also.

Fair enough.

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 64 at r1 (raw file):

Previously, shashidharatd (Shashidhara T D) wrote…

For federation use case. CoreDNS has to be deployed as normal k8s app and this is not default while installing helm chart. So i have given the overriding configuration to deploy CoreDNS as a normal k8s app with etcd as middleware.

Ok makes sense.

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 115 at r1 (raw file):

Previously, shashidharatd (Shashidhara T D) wrote…

Once the kube-dns configuration related PR is merged and available to users. users need not do this manual configuration at all. we should remove this section in future. So i feel we should leave out those details which we do take care of.

Ok.

Comments from Reviewable

madhusudancs · 2017-04-08T18:51:33Z

@devin-donnelly @chenopis @jaredbhatti could you please take a look?

madhusudancs · 2017-04-08T18:51:42Z

@shashidharatd this is also needs a rebase.

shashidharatd · 2017-04-09T16:26:13Z

@madhusudancs, sorry for the delay. rebased now

arthur0 · 2017-04-10T12:36:14Z

Hello @shashidharatd, I tried helm install --name etcd-operator stable/etcd-operator (with and without the --namespace flag) unsuccessfully. Do you know any reason for that?

Error: release etcd-operator failed: the server does not allow access to the requested resource (get namespaces default)

shashidharatd · 2017-04-10T14:07:51Z

Hello @arthur0,
I am able to install the chart with/without namespace successfully. i am suspecting you are having issue with RBAC settings in your k8s cluster.
Anyway it would be good to post this issue in https://github.com/kubernetes/charts instead of here. there would be more users/developers to help out.

csbell · 2017-04-12T18:45:54Z

Another possibility: federation doesn't create a default namespace. The failing "gets" suggest that might be the case.

chenopis

Some minor grammar nits.

chenopis · 2017-04-12T23:18:32Z

docs/tutorials/federation/set-up-coredns-provider-federation.md

+{:toc}
+
+This guide explains how to configure and deploy CoreDNS to be used as
+DNS provider for Cluster Federation.


add "the" before "DNS provider..."

chenopis · 2017-04-12T23:20:24Z

docs/tutorials/federation/set-up-coredns-provider-federation.md

+- shashidharatd
+title: Setting up CoreDNS as DNS provider for Cluster Federation
+---
+


If you could please use the Tutorial Template, that would really help us out. Thanks!

chenopis · 2017-04-12T23:21:00Z

docs/tutorials/federation/set-up-coredns-provider-federation.md

+This guide explains how to configure and deploy CoreDNS to be used as
+DNS provider for Cluster Federation.
+
+CoreDNS can be deployed in various configuration suiting the cluster


"configuration" -> "configurations"

chenopis · 2017-04-12T23:21:25Z

docs/tutorials/federation/set-up-coredns-provider-federation.md

+DNS provider for Cluster Federation.
+
+CoreDNS can be deployed in various configuration suiting the cluster
+federation deployment. In this guide we are explaining CoreDNS


The sentence "In this guide..." is redundant and can be removed because the points it makes are already mentioned above.

chenopis · 2017-04-12T23:25:11Z

docs/tutorials/federation/set-up-coredns-provider-federation.md

+
+## Deploying CoreDNS and etcd charts
+
+To deploy CoreDNS we shall make use of helm charts. CoreDNS will be


add comma: "To deploy CoreDNS, we shall..."

chenopis · 2017-04-12T23:37:23Z

docs/tutorials/federation/set-up-coredns-provider-federation.md

+
+## Deploying Federation with CoreDNS as DNS provider
+
+Federation control plane can be deployed using `kubefed init`. CoreDNS


add "the": "The Federation control plane..."

chenopis · 2017-04-12T23:37:47Z

docs/tutorials/federation/set-up-coredns-provider-federation.md

+## Deploying Federation with CoreDNS as DNS provider
+
+Federation control plane can be deployed using `kubefed init`. CoreDNS
+can be chosen as DNS provider by specifying two additional parameters.


add "the": "...can be chosen as the DNS provider..."

chenopis · 2017-04-12T23:39:05Z

docs/tutorials/federation/set-up-coredns-provider-federation.md

+
+Once the federation control plane is deployed and federated clusters
+are joined to the federation, you need to add the CoreDNS server to
+pod's nameserver resolv.conf chain in all the federated clusters as this


add "the": "...the pod's nameserver..."

chenopis · 2017-04-12T23:39:50Z

docs/tutorials/federation/set-up-coredns-provider-federation.md

+
+Replace `example.com` above with federation domain.
+
+> Note: Adding CoreDNS server to pod's nameserver resolv.conf chain will be


add "the": "...server to the pod's nameserver..."

chenopis · 2017-04-12T23:40:25Z

docs/tutorials/federation/set-up-coredns-provider-federation.md

+automated in subsequent releases.
+
+
+Now the federated cluster is ready for cross-cluster service dicovery !


typo and remove extra space: "...cross-cluster service discovery!"

shashidharatd · 2017-04-13T07:11:53Z

@chenopis, i have addressed your comments. PTAL

Review status: 1 of 3 files reviewed at latest revision, 21 unresolved discussions.

docs/tutorials/federation/set-up-coredns-provider-federation.md, line 6 at r3 (raw file):