Respect custom cache function

[mmorhun]

Problem description

For standard controller runtime it's possible to define cache function that would control what object kinds are cached partially by a selector. This is extremely useful if operator deals with a kind that have many instances in the cluster (usually Secrets and ConfigMaps), so caching them all would require a lot of memory and often causes OOM.

In KCP controller runtime this mechanism does not work.

Expected behavior

Cluster aware manager respects cache function option.

How to reproduce

options := ctrl.Options{
    ...
    NewCache: cache.BuilderWithOptions(cache.Options{
        SelectorsByObject: cache.SelectorsByObject{
                // The list of objects and the corresponding selectors are arbitrary
                // Here is a possible use-case
                &taskrunapi.PipelineRun{}: {
			Label: appStudioComponentPipelineRunSelector,
		},
		&corev1.Secret{}: {
			Label: partOfAppStudioSelector,
		},
        }
    }
}

...
mgr, err = kcp.NewClusterAwareManager(cfg, options)
...
mgr.Start(ctx)

Full example could be found here. Just uncomment the NewCache option.

Other notes

There is a hack that possibly could make it work, but:

• it requires changing internal URL which is unsafe and error prone
• it brings problems to the parts that already respect the path (so some functionality will not work as a consequence)
• it requires some reordering in code which is undesirable, especially if we need to support non-KCP clusters by the operator.

[fgiloux]

Hi Mykola

there is a wrapper for it. You need to use ClusterAwareBuilderWithOptions

[mmorhun]

Thank you, @fgiloux , the following worked for me:

options.NewCache = kcp.ClusterAwareBuilderWithOptions(cache.Options{
		SelectorsByObject: cache.SelectorsByObject{
                        &taskrunapi.PipelineRun{}: {
	        		Label: appStudioComponentPipelineRunSelector,
        		},
                }
})
kcp.NewClusterAwareManager(cfg, options)