feat(cli): migrate kingpin to cobra - dual parity

[TobyTheHutt]

What does it do ?

• Enforce Cobra vs Kingpin parity by validating required flags
• Test binder parity
• Ensure flag maintainability and extendibility
• Standardize Regex flag handling across both backends (Cobra & Kingpin)
• Cobra backend no longer supports --no-flags and instead requires the flag to be set to false

Motivation

Prepare for a migration from Kingpin to Cobra, as suggested in #5379 without breaking user experience.

More

• Yes, this PR title follows Conventional Commits
• Yes, I added unit tests
• Yes, I updated end user documentation accordingly

For the moment, full Kingpin functionality is ensured by default. The Cobra backend can be tested using the ENV variable EXTERNAL_DNS_CLI=cobra or hidden flag --cli-backend=cobra.

Continuous maintainability of flags is ensured through the FlagBinder which is picked up by both backends.

Next steps would likely be to invite operators to try out the new backend and to document the backend toggle.

[k8s-ci-robot]

Hi @TobyTheHutt. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[ivankatliarchuk]

Could you share evidence that the code is still working?

[ivankatliarchuk]

/ok-to-test

[TobyTheHutt]

Could you share evidence that the code is still working?

I won't be able to test all providers, if this is the question. However, I did test to my best skills and knowledge.

Following is the config to reproduce the tests, using a local etcd and coredns instance in docker:

docker network create extdns

# Create etcd instance
docker run -d --name extdns-etcd --network extdns -p 2379:2379 -p 2380:2380 \
  quay.io/coreos/etcd:v3.5.15 /usr/local/bin/etcd \
  --name s1 \
  --data-dir /etcd-data \
  --listen-client-urls http://0.0.0.0:2379 \
  --advertise-client-urls http://extdns-etcd:2379 \
  --listen-peer-urls http://0.0.0.0:2380 \
  --initial-advertise-peer-urls http://extdns-etcd:2380 \
  --initial-cluster s1=http://extdns-etcd:2380

# Create Corefile for coredns instance:
cat > Corefile <<'EOF'
example.test:1053 {
  errors
  log
  etcd example.test {
    stubzones
    path /skydns
    endpoint http://extdns-etcd:2379
  }
  forward . 1.1.1.1 8.8.8.8
  cache 30
}
EOF

# Create coredns instance:
docker run -d --name extdns-coredns --network extdns \
  -p 1053:1053/udp -p 1053:1053/tcp \
  -v "$PWD/Corefile:/Corefile:ro" \
  coredns/coredns:1.11.1 -conf /Corefile


# Run externalDNS to point to my local kind cluster using Kingpin backend:
export ETCD_URLS=http://127.0.0.1:2379
./build/external-dns \
  --provider=coredns \
  --source=ingress --source=service \
  --domain-filter=example.test \
  --policy=upsert-only \
  --registry=txt --txt-owner-id=localdev \
  --kubeconfig="$HOME/.kube/config" \
  --log-level=debug \
  --dry-run

# Run externalDNS to point to my local kind cluster using Cobra backend:
export ETCD_URLS=http://127.0.0.1:2379
./build/external-dns \
  --provider=coredns \
  --source=ingress --source=service \
  --domain-filter=example.test \
  --policy=upsert-only \
  --registry=txt --txt-owner-id=localdev \
  --kubeconfig="$HOME/.kube/config" \
  --log-level=debug \
  --dry-run
  --cli-backend=cobra

The result:

With Kingpin:

INFO[0001] config: {APIServerURL: KubeConfig:/home/tobi-wan/.kube/config RequestTimeout:30s DefaultTargets:[] GlooNamespaces:[gloo-system] SkipperRouteGroupVersion:zalando.org/v1 Sources:[ingress service] Namespace: AnnotationFilter: LabelFilter: IngressClassNames:[] FQDNTemplate: CombineFQDNAndAnnotation:false IgnoreHostnameAnnotation:false IgnoreNonHostNetworkPods:false IgnoreIngressTLSSpec:false IgnoreIngressRulesSpec:false ListenEndpointEvents:false ExposeInternalIPV6:false GatewayName: GatewayNamespace: GatewayLabelFilter: Compatibility: PodSourceDomain: PublishInternal:false PublishHostIP:false AlwaysPublishNotReadyAddresses:false ConnectorSourceServer:localhost:8080 Provider:coredns ProviderCacheTime:0s GoogleProject: GoogleBatchChangeSize:1000 GoogleBatchChangeInterval:1s GoogleZoneVisibility: DomainFilter:[example.test] ExcludeDomains:[] RegexDomainFilter: RegexDomainExclusion: ZoneNameFilter:[] ZoneIDFilter:[] TargetNetFilter:[] ExcludeTargetNets:[] AlibabaCloudConfigFile:/etc/kubernetes/alibaba-cloud.json AlibabaCloudZoneType: AWSZoneType: AWSZoneTagFilter:[] AWSAssumeRole: AWSProfiles:[] AWSAssumeRoleExternalID: AWSBatchChangeSize:1000 AWSBatchChangeSizeBytes:32000 AWSBatchChangeSizeValues:1000 AWSBatchChangeInterval:1s AWSEvaluateTargetHealth:true AWSAPIRetries:3 AWSPreferCNAME:false AWSZoneCacheDuration:0s AWSSDServiceCleanup:false AWSSDCreateTag:map[] AWSZoneMatchParent:false AWSDynamoDBRegion: AWSDynamoDBTable:external-dns AzureConfigFile:/etc/kubernetes/azure.json AzureResourceGroup: AzureSubscriptionID: AzureUserAssignedIdentityClientID: AzureActiveDirectoryAuthorityHost: AzureZonesCacheDuration:0s AzureMaxRetriesCount:3 CloudflareProxied:false CloudflareCustomHostnames:false CloudflareDNSRecordsPerPage:100 CloudflareDNSRecordsComment: CloudflareCustomHostnamesMinTLSVersion:1.0 CloudflareCustomHostnamesCertificateAuthority:none CloudflareRegionalServices:false CloudflareRegionKey: CoreDNSPrefix:/skydns/ AkamaiServiceConsumerDomain: AkamaiClientToken: AkamaiClientSecret: AkamaiAccessToken: AkamaiEdgercPath: AkamaiEdgercSection: OCIConfigFile:/etc/kubernetes/oci.yaml OCICompartmentOCID: OCIAuthInstancePrincipal:false OCIZoneScope:GLOBAL OCIZoneCacheDuration:0s InMemoryZones:[] OVHEndpoint:ovh-eu OVHApiRateLimit:20 OVHEnableCNAMERelative:false PDNSServer:http://localhost:8081 PDNSServerID:localhost PDNSAPIKey: PDNSSkipTLSVerify:false TLSCA: TLSClientCert: TLSClientCertKey: Policy:upsert-only Registry:txt TXTOwnerID:localdev TXTPrefix: TXTSuffix: TXTEncryptEnabled:false TXTEncryptAESKey: Interval:1m0s MinEventSyncInterval:5s MinTTL:0s Once:false DryRun:true UpdateEvents:false LogFormat:text MetricsAddress::7979 LogLevel:debug TXTCacheInterval:0s TXTWildcardReplacement: ExoscaleEndpoint: ExoscaleAPIKey: ExoscaleAPISecret: ExoscaleAPIEnvironment:api ExoscaleAPIZone:ch-gva-2 CRDSourceAPIVersion:externaldns.k8s.io/v1alpha1 CRDSourceKind:DNSEndpoint ServiceTypeFilter:[] CFAPIEndpoint: CFUsername: CFPassword: ResolveServiceLoadBalancerHostname:false RFC2136Host:[] RFC2136Port:0 RFC2136Zone:[] RFC2136Insecure:false RFC2136GSSTSIG:false RFC2136CreatePTR:false RFC2136KerberosRealm: RFC2136KerberosUsername: RFC2136KerberosPassword: RFC2136TSIGKeyName: RFC2136TSIGSecret: RFC2136TSIGSecretAlg: RFC2136TAXFR:false RFC2136MinTTL:0s RFC2136LoadBalancingStrategy:disabled RFC2136BatchChangeSize:50 RFC2136UseTLS:false RFC2136SkipTLSVerify:false NS1Endpoint: NS1IgnoreSSL:false NS1MinTTLSeconds:0 TransIPAccountName: TransIPPrivateKeyFile: DigitalOceanAPIPageSize:50 ManagedDNSRecordTypes:[A AAAA CNAME] ExcludeDNSRecordTypes:[] GoDaddyAPIKey: GoDaddySecretKey: GoDaddyTTL:0 GoDaddyOTE:false OCPRouterName: PiholeServer: PiholePassword: PiholeTLSInsecureSkipVerify:false PiholeApiVersion:5 PluralCluster: PluralProvider: WebhookProviderURL:http://localhost:8888 WebhookProviderReadTimeout:5s WebhookProviderWriteTimeout:10s WebhookServer:false TraefikEnableLegacy:false TraefikDisableNew:false NAT64Networks:[] ExcludeUnschedulable:true EmitEvents:[] ForceDefaultTargets:false sourceWrappers:map[]} 
INFO[0001] running in dry-run mode. No changes to DNS records will be made. 
INFO[0001] GitCommitShort=649fff89, GoVersion=go1.24.6, Platform=linux/amd64, UserAgent=ExternalDNS/649fff89 
INFO[0001] Instantiating new Kubernetes client          
DEBU[0001] apiServerURL:                                
DEBU[0001] kubeConfig: /home/tobi-wan/.kube/config 
INFO[0001] Using kubeConfig                             
DEBU[0001] serving 'healthz' on ':7979/healthz'         
DEBU[0001] serving 'metrics' on ':7979/metrics'         
DEBU[0001] registered '21' metrics                      
INFO[0001] Created Kubernetes client https://kubernetes.docker.internal:6443 
DEBU[0001] dedupSource: collecting endpoints and removing duplicates 
DEBU[0001] multiSource: collecting endpoints from 2 child sources and removing duplicates 
DEBU[0001] No endpoints could be generated from service default/kubernetes 
DEBU[0001] No endpoints could be generated from service kube-system/kube-dns 
INFO[0001] All records are already up to date

With Cobra:

INFO[0002] config: {APIServerURL: KubeConfig:/home/tobi-wan/.kube/config  RequestTimeout:30s DefaultTargets:[] GlooNamespaces:[gloo-system] SkipperRouteGroupVersion:zalando.org/v1 Sources:[ingress service] Namespace: AnnotationFilter: LabelFilter: IngressClassNames:[] FQDNTemplate: CombineFQDNAndAnnotation:false IgnoreHostnameAnnotation:false IgnoreNonHostNetworkPods:false IgnoreIngressTLSSpec:false IgnoreIngressRulesSpec:false ListenEndpointEvents:false ExposeInternalIPV6:false GatewayName: GatewayNamespace: GatewayLabelFilter: Compatibility: PodSourceDomain: PublishInternal:false PublishHostIP:false AlwaysPublishNotReadyAddresses:false ConnectorSourceServer:localhost:8080 Provider:coredns ProviderCacheTime:0s GoogleProject: GoogleBatchChangeSize:1000 GoogleBatchChangeInterval:1s GoogleZoneVisibility: DomainFilter:[example.test] ExcludeDomains:[] RegexDomainFilter: RegexDomainExclusion: ZoneNameFilter:[] ZoneIDFilter:[] TargetNetFilter:[] ExcludeTargetNets:[] AlibabaCloudConfigFile:/etc/kubernetes/alibaba-cloud.json AlibabaCloudZoneType: AWSZoneType: AWSZoneTagFilter:[] AWSAssumeRole: AWSProfiles:[] AWSAssumeRoleExternalID: AWSBatchChangeSize:1000 AWSBatchChangeSizeBytes:32000 AWSBatchChangeSizeValues:1000 AWSBatchChangeInterval:1s AWSEvaluateTargetHealth:true AWSAPIRetries:3 AWSPreferCNAME:false AWSZoneCacheDuration:0s AWSSDServiceCleanup:false AWSSDCreateTag:map[] AWSZoneMatchParent:false AWSDynamoDBRegion: AWSDynamoDBTable:external-dns AzureConfigFile:/etc/kubernetes/azure.json AzureResourceGroup: AzureSubscriptionID: AzureUserAssignedIdentityClientID: AzureActiveDirectoryAuthorityHost: AzureZonesCacheDuration:0s AzureMaxRetriesCount:3 CloudflareProxied:false CloudflareCustomHostnames:false CloudflareDNSRecordsPerPage:100 CloudflareDNSRecordsComment: CloudflareCustomHostnamesMinTLSVersion:1.0 CloudflareCustomHostnamesCertificateAuthority:none CloudflareRegionalServices:false CloudflareRegionKey: CoreDNSPrefix:/skydns/ AkamaiServiceConsumerDomain: AkamaiClientToken: AkamaiClientSecret: AkamaiAccessToken: AkamaiEdgercPath: AkamaiEdgercSection: OCIConfigFile:/etc/kubernetes/oci.yaml OCICompartmentOCID: OCIAuthInstancePrincipal:false OCIZoneScope:GLOBAL OCIZoneCacheDuration:0s InMemoryZones:[] OVHEndpoint:ovh-eu OVHApiRateLimit:20 OVHEnableCNAMERelative:false PDNSServer:http://localhost:8081 PDNSServerID:localhost PDNSAPIKey: PDNSSkipTLSVerify:false TLSCA: TLSClientCert: TLSClientCertKey: Policy:upsert-only Registry:txt TXTOwnerID:localdev TXTPrefix: TXTSuffix: TXTEncryptEnabled:false TXTEncryptAESKey: Interval:1m0s MinEventSyncInterval:5s MinTTL:0s Once:false DryRun:true UpdateEvents:false LogFormat:text MetricsAddress::7979 LogLevel:debug TXTCacheInterval:0s TXTWildcardReplacement: ExoscaleEndpoint: ExoscaleAPIKey: ExoscaleAPISecret: ExoscaleAPIEnvironment:api ExoscaleAPIZone:ch-gva-2 CRDSourceAPIVersion:externaldns.k8s.io/v1alpha1 CRDSourceKind:DNSEndpoint ServiceTypeFilter:[] CFAPIEndpoint: CFUsername: CFPassword: ResolveServiceLoadBalancerHostname:false RFC2136Host:[] RFC2136Port:0 RFC2136Zone:[] RFC2136Insecure:false RFC2136GSSTSIG:false RFC2136CreatePTR:false RFC2136KerberosRealm: RFC2136KerberosUsername: RFC2136KerberosPassword: RFC2136TSIGKeyName: RFC2136TSIGSecret: RFC2136TSIGSecretAlg: RFC2136TAXFR:false RFC2136MinTTL:0s RFC2136LoadBalancingStrategy:disabled RFC2136BatchChangeSize:50 RFC2136UseTLS:false RFC2136SkipTLSVerify:false NS1Endpoint: NS1IgnoreSSL:false NS1MinTTLSeconds:0 TransIPAccountName: TransIPPrivateKeyFile: DigitalOceanAPIPageSize:50 ManagedDNSRecordTypes:[A AAAA CNAME] ExcludeDNSRecordTypes:[] GoDaddyAPIKey: GoDaddySecretKey: GoDaddyTTL:0 GoDaddyOTE:false OCPRouterName: PiholeServer: PiholePassword: PiholeTLSInsecureSkipVerify:false PiholeApiVersion:5 PluralCluster: PluralProvider: WebhookProviderURL:http://localhost:8888 WebhookProviderReadTimeout:5s WebhookProviderWriteTimeout:10s WebhookServer:false TraefikEnableLegacy:false TraefikDisableNew:false NAT64Networks:[] ExcludeUnschedulable:true EmitEvents:[] ForceDefaultTargets:false sourceWrappers:map[]} 
INFO[0002] running in dry-run mode. No changes to DNS records will be made. 
INFO[0002] GitCommitShort=649fff89, GoVersion=go1.24.6, Platform=linux/amd64, UserAgent=ExternalDNS/649fff89 
DEBU[0002] serving 'healthz' on ':7979/healthz'         
DEBU[0002] serving 'metrics' on ':7979/metrics'         
DEBU[0002] registered '21' metrics                      
INFO[0002] Instantiating new Kubernetes client          
DEBU[0002] apiServerURL:                                
DEBU[0002] kubeConfig: /home/tobi-wan/.kube/config 
INFO[0002] Using kubeConfig                             
INFO[0003] Created Kubernetes client https://kubernetes.docker.internal:6443 
DEBU[0003] dedupSource: collecting endpoints and removing duplicates 
DEBU[0003] multiSource: collecting endpoints from 2 child sources and removing duplicates 
DEBU[0003] No endpoints could be generated from service default/kubernetes 
DEBU[0003] No endpoints could be generated from service kube-system/kube-dns 
INFO[0003] All records are already up to date

Did I go too far in my code changes, since I cannot test all provider flags? If so, I can still revert changes which I cannot test within reasonable effort and budget. Let me know how to best proceed.

I saw the race condition in the test and I'm on track to fix that too.

[TobyTheHutt]

The failing tests shouldn't be caused by my changes to the binders and flags. The binder package passed, validation passed and failures are isolated to source/wrappers, involving test helper mutating shared endpoints.

I currently cannot see a connection to my changes and the project wrappers.

Just to be sure, I ran the makefile tests in a throttled docker environment with 0.5 CPU, which succeeded.

/test pull-external-dns-unit-test

[ivankatliarchuk]

Yeah, there are data races, and the fix awaiting review #5833

[ivankatliarchuk]

No need to test all providers, just high level smoke test is enough. Need to make sure the code still executes ))

[mloiseleur]

@TobyTheHutt A first fix on the race condition has been merged yesterday (#5841). If you rebase, that should be better.

[ivankatliarchuk]

/lgtm

[mloiseleur]

/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mloiseleur

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [mloiseleur]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment