Support globbing in the plugin `PackagePlugin` library

[FranzBusch]

Description

Plugins often need to expose configs where users can pick what files to apply a plugin to. A common tool for doing this is to use globbing to filter for the right files. We just got a feature request for this over in apple/swift-protobuf#1311.

It would be great if the PackagePlugin library exposes methods where you can pass a String which contains globs to get files of a SourceTarget.

rdar://92865590

[tomerd]

cc @elsh @neonichu @abertelrud

[neonichu]

I think globs would be very useful in the package manifest which ideally should be declarative, but for plugins it seems more like a nice-to-have since it can be implemented by the plugin itself as well.

[FranzBusch]

@neonichu I agree that for the manifest itself it is more interesting; however, I would really like to avoid to implement it for every plugin. The full glob support including excludes etc. is quite some work to do.

I would love if we would provide this in a central way. One possible solution could be that there is a globbing package and if we allow plugins to have dependencies then we could solve it this way. In general, I just think that there is some work to-do in SPM to support this while avoiding every plugin to reimplement globbing.

[tomerd]

@FranzBusch do you have cycles to do a proof of concept for such globbing support? we can then figure out how to best expose it both at the manifest level and plugin, which both have an interesting use case of not having access to packages but need some sort of "SDK" provided by SwiftPM

[denil-ct]

I'd love to have a go at this, as adding individual files to protobuf's plugin config is quite tiring.
Can anyone point me in a direction on how to proceed?

@FranzBusch @tomerd

[tomerd]

cc @neonichu @abertelrud can we add an API that uses regex for this purpose?

[neonichu]

I think we really need to rewrite TargetSourcesBuilder first probably, otherwise globbing has the potential of making the performance even worse if we just tack it on. In terms of implementation, I would assume we use fnmatch instead of rolling our own thing based on regexes.

[denil-ct]

Can't we directly use glob from Darwin and glibc for mac and linux appropriately?

[neonichu]

Not 100% sure, but I think glob would traverse the file system again and what I would like to do is to change TargetSourcesBuilder to collect the file system information once and then apply rules and things like globs in memory to avoid performance issues.