Cannot render kpt package with batch/v1 Jobs using metadata.generateName

[zevisert]

Expected behavior

I can kpt fn render a package containing a Job that uses metadata.generateName instead of metadata.name using curated functions from catalog.kpt.dev.

Actual behavior

> kpt fn render aheqox
Package "aheqox": 
[RUNNING] "set-namespace:v0.4.1"
[FAIL] "set-namespace:v0.4.1" in 0s
Error: input resource list must contain only KRM resources: image-migration.job.yaml: resource must have `metadata.name` 

Information

I have a job that performs some migrations tasks that is to be run once immediately after deployment. Since deployments may occur one-after-another in quick succession, setting spec.ttlSecondsAfterFinished is insufficient, since the spec of the job being deployed gets changed with each deployment, leading to

> kpt live apply # using metadata.name on my job
Resource: "batch/v1, Resource=jobs", GroupVersionKind: "batch/v1, Kind=Job"
Name: "image-migration", Namespace: "project-aheqox"
for: "some.job.yaml": Job.batch "migrate-tasks" is invalid: spec.template: Invalid value: ...

> kpt version 
1.0.0-beta.13

Steps to reproduce the behavior

1. Create these files:

# ./Kptfile
apiVersion: kpt.dev/v1
kind: Kptfile
metadata:
  name: aheqox
  annotations:
    config.kubernetes.io/local-config: "true"
info:
  description: A KPT package demonstrating an issue with generateName.
pipeline:
  mutators:
    - image: set-namespace:v0.4.1
      configMap:
        namespace: project-aheqox
    - image: apply-setters:v0.2.0
      configMap:
        # Not fully relevant, but here to demonstrate that the job `.spec.template` may be changing,
        # which prevents using a metadata.name in our use case.
        new-name: example1 

# ./some.job.yaml
apiVersion: batch/v1
kind: Job
metadata:
  generateName: migrate-tasks-
spec:
  ttlSecondsAfterFinished: 3600
  template:
    spec:
      containers:
        - name: example # kpt-set: ${new-name}
          image: hello-world

2. Run kpt fn render .

[zevisert]

Possibly related, but not too sure. It's just the only other mention of generateName that I could find: #3222

[droot]

Thank you reporting this @zevisert

This is indeed a very interesting edge case that's breaking the assumption that all resources will have name.

Treating metadata.generateName if metadata.Name is missing could probably work but will have ripple effect across the kpt stack, so will have to think through it.

/cc @yuwenma This is interesting case, any ideas to support such resources ?

[yuwenma]

Hi @zevisert. Thank you for filing the issue.

Not supporting generateName is actually an expected behavior I believe. Here're reasons why:
generateName does not work coherently with declarative operations in general, thus commands like kubectl apply (see issue) and kustomize (see issue) both choose not to support it.

Kpt is designed to be declarative, KRM resource centric and shifting the reliable KRM resource editing on the client side, it even keeps track of the KRM resource's upstream Group, Version, Namespace and Name in the upstream-identifier annotation to guarantee the function idempotent between upstream/downstream. If you use generateName, it means the name will be completed in the apply time, and the KRM functions can no longer bundle the Job object with other Job related resources.

[johnbelamaric]

Before closing this, can we spend a little more time and help @zevisert understand how they might solve the issue using the kpt toolchain? Depending on the details of the use case, a few options come to mind:

1. Is there a reason migration tasks are not in an init-container? That would be the more typical flow for a run-once-per pod type of case.
2. A function could generate the name and replace it (ie, we could have a placeholder name in the job and do the generateName functionality client side)

There may be other solutions, depending on the details. When you say "run once immediately after deployment", can you clarify the deployment of what exactly?

[yuwenma]

+1 to the iniContainer approach.
@zevisert I'll reopen this issue and please let us know how that works. We are happy to learn more about your use cases.

[zevisert]

Thanks for the discussion everyone, and thanks for reopening to help us solve this!

I totally understand how generateName would break the declarative model - I don't really need the solution to be via generateName. I'd just like to be able to deploy a Job more often than Job:spec.ttlSecondsAfterFinished, all the while keeping that value relatively high (several hours) so that we can see logs and status without going "off-cluster".

---

When you say "run once immediately after deployment", can you clarify the deployment of what exactly?

I mean package deployment, a-la kpt live apply - there's a variety of typical resources in our package, actual apps/v1:Deployments, but other common stuff too, ConfigMaps, Services, etc.

---

We picked Job instead of an init container on our deployment because of RBAC concerns. If the migration we need to apply were to be an init container, our entire deployment would need elevated permissions, which wasn't really an option for us (unless I'm mistaken, service accounts apply to the whole pod).

But you've lead me to a new solution to try, we could try using an init container on a throwaway deployment who's main workload exits right away and uses a restartPolicy: never. Edit: TIL, Deployments require restartPolicy: Always. Oh, and ReplicaSets too!

[johnbelamaric]

I would generate the name client side then, with a function.

[zevisert]

Could do that pretty easily, sure. Checking my understanding, we're fine to do that inventory-wise right? The previously generated job name would get pruned from the inventory and deleted from the cluster if it's still there, and the new name gets saved in the inventory, right. That would seem pretty good to me.

Edit; yeah of course, that's how kpt works :)

[zevisert]

Thanks @johnbelamaric, I got caught up thinking about init-containers with that writeup, generating a name client side should work just fine.