cache contained <nil>

[DerekTBrown]

I am using controller-runtime to create a controller that tracks pods using a cache. I get the following error:

failed to list from informer: cache contained <nil>, which is not an Object

This is coming from the following line, which indicates that the condition should never happen:

https://github.com/kubernetes-sigs/controller-runtime/blob/main/pkg/cache/internal/cache_reader.go#L80

How should I proceed to debug this?

[DerekTBrown]

Possibly related to: actions/actions-runner-controller#776

[DerekTBrown]

[Correction] this is coming from a List call, so its actually coming from:

https://github.com/kubernetes-sigs/controller-runtime/blob/v0.18.4/pkg/cache/internal/cache_reader.go#L153

[alvaroaleman]

I've never seen this. Are you using UnsafeDisableDeepCopy?

[DerekTBrown]

Are you using UnsafeDisableDeepCopy?

Nope.

[DerekTBrown]

I've never seen this.

I am having an incredibly hard time trying to reproduce this. I tried introspecting controller-runtime and client-go, but I can't see any obvious race conditions or anything that would lead to this behavior.

To try to bisect the issue further, I created a patched version of client-go that will panic if ever a nil object becomes present in the cache:

DerekTBrown/client-go#1

I then patched this into my controller, but haven't seen the issue re-appear. Notably, when I did the patching, I saw a bunch of unrelated version upgrades in my go.sum:

I wonder if this is a bug somewhere in one of these older package versions.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten