Repository navigation
Migrate to the _v1 kubernetes resource names before provider v4 removes the old ones #253
Description
What
The hub uses the deprecated non-suffixed hashicorp/kubernetes resource types everywhere — 197
occurrences across the modules. Plans now print warnings like Deprecated; use kubernetes_service_account_v1, first noticed in the step logs of an ske/forgejo-connector
building block run.
Counts by type:
67 kubernetes_secret
40 kubernetes_service_account
32 kubernetes_cluster_role_binding
26 kubernetes_cluster_role
23 kubernetes_role_binding
5 kubernetes_namespace
4 kubernetes_default_service_account
Why this is not just noise
The provider documents that the plain names are removed in v4.0.0
(https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/guides/versioned-resources).
v3.0.0 already deprecated all seven types above. Our building block modules pin the provider with an
open upper bound — three modules say >= 2.35.1 and one says >= 2.38 — so v4 will resolve by
itself on the day it ships. Every deployed building block instance then breaks at plan with Invalid resource type, with no warning phase and no rollback for instances that are already out there.
The rename is not a rename
A moved block cannot do it. All of these types are SDKv2-backed in hashicorp/kubernetes, and
SDKv2 refuses the MoveResourceState RPC unconditionally (helper/schema/grpc_provider.go, "Move
Resource State Not Supported"), so a cross-type moved block fails at plan on every Terraform and
OpenTofu version. This works for our own meshstack_tenant_v4 to meshstack_tenant moves only
because the meshstack provider is plugin-framework based and implements MoveState.
The only non-destructive path the provider documents is terraform state rm followed by
terraform import.
Changing the type names without that gives a destroy and create. For one existing forgejo connector
that means:
kubernetes_secret.forgejo_actionsis a service account token, so the kubeconfig rotates. It feeds
local.action_secrets, which is hashed intoterraform_data.await_pipeline_workflow.triggers_replace,
so the connector re-triggers the Forgejo pipeline and blocks up to 15 minutes waiting for
build_image,deploy— and fails the building block run if that pipeline fails.kubernetes_default_service_account.thisinherits its delete from the plain service account
resource, so it really deletes the namespace'sdefaultServiceAccount. Kubernetes recreates a bare
one and the create step re-patchesimagePullSecrets. Pods started in that window get
ImagePullBackOff, and the re-triggereddeployjob runs in exactly that window.- The role binding and cluster role binding are destroyed before they are recreated, so there is a gap
with noeditpermission in the namespace.
The lever we do have: the pre-run script
implementation.terraform.pre_run_script on a building block definition version runs after the
"Initialize Terraform and select Workspace" step, so the backend is already configured and the state
is reachable from the runner. modules/stackit/project already uses one (prerun.sh exec'ing
prerun.py).
That makes the documented state rm plus import migration workable inside meshStack, which it is
not from an operator machine — the http backend gives nobody CLI access to that state. A migration
script would, per resource:
- Check whether the object is already tracked under the
_v1address, and do nothing if so, so the
script is idempotent across re-runs. tofu state rm <old address>.tofu import <new address> <id>with the same id the resource already has.
It needs to be a no-op for building block instances created fresh off the migration ref, since they
have nothing to migrate. Once every instance has run it, a later ref drops the script.
Proposed steps
- Now, separately from any rename: pin the kubernetes provider to
~> 3.0in every module that
declares it. One line each, no plan change, and it turns "everything breaks the day v4 ships" into
a decision we schedule. - Write new modules with
_v1names from the start. - Treat the 197 existing occurrences as a v4 upgrade project, module by module, using a pre-run
migration script.ske/forgejo-connectoris the natural first one, since it is where the warnings
were noticed and it has eight resource blocks across six types.
Activity