Skip to content

Implement CoreDNS ConfigMap integration in the dns-controller #348

Description

Summary

The dns-controller README at cmd/atenet/internal/dns/README.md lists two integration paths for wiring substrate's stub resolver into the cluster's DNS:

## Integration

* CoreDNS: Update CoreDNS ConfigMap to add the stub resolver.
* GKE DNS: Update the GKE DNS ConfigMap to add the stub resolver.

The GKE path lives in reconcileKubeDNSConfig today (writes stubDomains JSON into kube-system/kube-dns). Implementing the CoreDNS path would let substrate's actor hostnames resolve on any cluster using a stock CoreDNS Corefile — kubeadm-based clusters (kind, k3s, vanilla self-managed), EKS, AKS, on-prem, and GKE installs that don't surface the kube-dns compatibility ConfigMap.

Motivation

On a stock kubeadm cluster the cluster DNS is CoreDNS configured by a kube-system/coredns ConfigMap whose Corefile key holds the CoreDNS DSL:

data:
  Corefile: |
    .:53 {
        errors
        kubernetes cluster.local in-addr.arpa ip6.arpa { ... }
        forward . /etc/resolv.conf { ... }
        cache 30
        loop
        reload
        loadbalance
    }

After installing substrate, the dns-controller logs:

kube-dns ConfigMap not found in kube-system namespace, skipping stub resolver configuration

…and the cluster's CoreDNS never learns where to forward actors.resources.substrate.ate.dev. Adding the CoreDNS path closes the loop for these clusters without operator intervention.

Proposed implementation

A peer to reconcileKubeDNSConfig that targets kube-system/coredns, gated by a new flag on atenet dns:

  1. Selection. Add --cluster-dns-style=kube-dns|coredns to the atenet dns subcommand, defaulting to kube-dns so existing GKE deployments are unaffected. Operators on kubeadm-based clusters pass --cluster-dns-style=coredns. Auto-detection can come later as a separate enhancement if useful — keeping the first cut explicit is simpler to review and reason about.

  2. Editing the Corefile idempotently. Use a CoreDNS Corefile parser (e.g. github.com/coredns/caddy/caddyfile) to load the AST, splice in / update / remove a substrate-owned server block, and serialize back. Avoids text/regex munging on a shared cluster-infrastructure ConfigMap.

  3. Stub zone to insert. A peer server block sibling to the existing .:53, marked so future reconciles can find and update it idempotently. Conceptually:

    # managed by substrate dns-controller
    actors.resources.substrate.ate.dev:53 {
        errors
        cache 30
        forward . <substrate-dns-clusterIP>
    }
    
    .:53 { ... }   # untouched
    
  4. Reload. Write the updated ConfigMap. Stock kubeadm CoreDNS uses the reload plugin which watches the mounted Corefile and reloads on change, so no extra signaling needed.

  5. Removal. On uninstall (or when the substrate CoreDNS Service goes away), remove the substrate-owned block cleanly so the cluster Corefile returns to its prior state.

Risk surface

kube-system/coredns is shared cluster infrastructure — the implementation should:

  • Touch only the substrate-owned server block; never modify the default .:53 block.
  • Be idempotent — repeated reconciles must converge to the same Corefile.
  • Tolerate a hand-edited Corefile elsewhere in the file — leave non-substrate content alone.
  • Fail soft on a malformed Corefile (log + skip a reconcile rather than rewrite).

Repro on kind

./hack/install-ate-kind.sh
# wait for substrate components to come up

# Create an actor (any WorkerPool / ActorTemplate flow) and attempt to dial
# <actor-id>.actors.resources.substrate.ate.dev from a pod -> NXDOMAIN.

kubectl get cm -n kube-system coredns -o yaml
# Corefile has no substrate stub.

kubectl -n ate-system logs deploy/dns -c dns-controller
# "kube-dns ConfigMap not found in kube-system namespace, skipping stub resolver configuration"

Contribution

Happy to take this on if the direction sounds right.

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions