Enable globbing in "resources" field of kustomization.yaml

[yanniszark]

This is basically the same as #119.
Reopening, as the issue was closed.
The reason for reopening is that we (Arrikto) believe that the reasons given for eschewing this feature don't really apply.

It's also interesting that other use-cases mandating glob-like use have come up from the Kustomize team (although I feel the particular one could benefit from more explicit globbing):
#3204

Is your feature request related to a problem? Please describe.

I want to automatically pick up new kustomizations inside a folder. For example, we have a structure like this:

user-resources
├── user1
└── user2
...

and I want to have a kustomization file that automatically imports everything under user-resources:

resources:
- path/to/user-resources/*

Describe the solution you'd like

Kustomize should support globbing in resources. This feature is eschewed right now and the given reason is:

kustomize supports the best practice of storing one’s entire configuration in a version control system.

Globbing the local file system for files not explicitly declared in the kustomization file at kustomize build time would violate that goal.

Allowing globbing in a kustomization file would also introduce the same problems as allowing globbing in java import declarations or BUILD/Makefile dependency rules.

kustomize will instead provide kustomization file editting commands that accept globbed arguments, expand them at edit time relative to the local file system, and store the resulting explicit names into the kustomization file.

I'd like to elaborate why we are not very clear on those arguments.

kustomize supports the best practice of storing one’s entire configuration in a version control system.

Globbing the local file system for files not explicitly declared in the kustomization file at kustomize build time would violate that goal.

This is not clear to us. Storing the entire configuration in a VCS means that checking out a specific commit and running kustomize build should produce the same result everytime. That's why kustomize has explicitly avoided CLI arguments and environment variables. But in the case of globs, the result of globbing the filesystem of a specific commit will always be the same. There is no hidden state here.

Allowing globbing in a kustomization file would also introduce the same problems as allowing globbing in java import declarations or BUILD/Makefile dependency rules.

The problem mentioned in the referenced blog post is that when importing with globs, name collisions may occur. The example given in the blog post is that if one writes:

import java.util.*;
import java.awt.*;

// Where did the List class come from?
List choices = new List();

and the List class is declared by two imports, Java will silently keep the one declared later in the code.

However, this doesn't really apply to kustomize, because:

• Kustomize forbids resource collision (same group, version, kind, namespace, name).
• Kustomize imports are not very useful for deterring collisions. If one follows the suggested best practice and imports both java.util.List and java.awt.List, then it's immediately clear there is a collision, so catastrophe avoided. However, if one imports a/deployment.yaml and b/deployment.yaml, it's not really clear if there is a resource collision.

Thus, we believe this drawback doesn't really exist for a tool like kustomize.

Describe alternatives you've considered

Kustomize provides the alternative of CLI commands that support globbing and can refresh the kustomizations.
However, this imposes an additional burden on the user, to track every single place where this must occur.

I'd love to hear your thoughts on this @monopole @Liujingfang1 @Shell32-Natsu
Also cc'ing the author of the original issue @ahmetb

[monopole]

It's become clear over time that kustomize is primarily used in a git context.

When that's the case, questions about what went into a build are moot, and a resources: field can be globbed or even eliminated as suggested in #3204.

[monopole]

Let's make sure kustomize edit add resources .... accepts a list.

[monopole]

Also update docs site to describe how to use globs to manage the goal here.

[rptaylor]

I have to say I am surprised by this issue. I just started learning to use kustomize yesterday and it seemed like it would perfectly fit in my workflow. I have a directory of all my YAML files in git and I simply kubectl apply -f myyaml/ on the directory to recursively apply all the YAML there, and kustomize in kubectl seems exactly like a drop-in alternative kubectl apply -k myyaml/ but with the benefit of consolidating and simplifying my YAML management.

It seemed so intuitive and obvious that

bases:
  - ../baseyaml

should apply all the YAML in that directory, just like kubectl apply -f , that I was very confused about why you would need to explicitly specify any .yml files as resources in kustomization.yml in the base directory since this would be redundant. This is in all the examples but the documentation does not explain why. I thought you needed to specify a resource in the base layer only if you want to allow a higher layer to overwrite those values, and that otherwise it would be "read only" (but obviously pulled in by default to the upper layers) because I could not think of any other way this would make sense. It was so counterintuitive it took awhile to figure out how mislead I was.

So, now I download yet another tool (kustomize) , dig up an ancient version to avoid #1342 so it doesn't break kubectl apply -k , run another command (kustomize edit add resource *.yml) , clean up after it because *.yml matches kustomization.yml and it didn't know not to add itself as a resource in kustomization.yml , and then I end up with uncommitted changes made to kustomization.yml in my version control as a result of this extraneous process (so it does not really seem that conducive to a git-based workflow IMHO), when all I wanted was to do what is (from my point of view) obviously implied by specifying a directory as a base in the first place: just use the YAML in that directory. In hindsight I should have just scripted up something dumb like this: #119 (comment)

kustomize supports the best practice of storing one’s entire configuration in a version control system.

Globbing the local file system for files not explicitly declared in the kustomization file at kustomize build time would violate that goal.

From my point of view I want a way to explicitly declare a directory to use as a base. And when I manage the contents of that directory in version control, it is not at all clear how loading all the YAML there would violate the best practice of storing one’s entire configuration in a version control system (in fact precisely the opposite, it seems to closely conform to that goal).
It is my responsibility to make sure I have the right YAML contents in my version control , not kustomize's responsibility.
kustomize should not have opinions about how I specify or manage my files; it should do what I say , like any tool. If I accidentally break my deployment I will blame myself but if kustomize doesn't let me easily manage and apply YAML maybe I should blame kustomize :)
Well I won't blame it , I made some incorrect assumptions and my intuition and expectations may have been misguided, but I hope it may be productive to describe my initial (maybe naive) user experience and perspective running into this issue. Thank you!

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale

[Shell32-Natsu]

/remove-lifecycle stale

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale

[rptaylor]

/remove-lifecycle stale