sdnotify play kube policies: podman container wait, hangs

[edsantiago]

[+1368s] not ok 310 sdnotify : play kube - with policies
...
$ podman container inspect test_pod-a --format {{.Config.SdNotifySocket}}
/var/tmp/-podman-notify-proxy.sock54032352
$ podman logs test_pod-a
/run/notify/notify.sock
READY
$ podman exec test_pod-a /bin/touch /stop
$ podman container inspect 9cffc8377ee1-service --format {{.State.ConmonPid}}
98526
$ podman container wait test_pod-a
timeout: sending signal TERM to command ?/var/tmp/go/src/github.com/containers/podman/bin/podman?
[ rc=124 (** EXPECTED 0 **) ]
*** TIMED OUT ***

• fedora-36 : sys podman fedora-36 rootless host
  • PR Update vendor of containers/buildah v1.28.0 #16034
    • 10-03 15:32
    • 10-03 11:37
• fedora-36 : sys podman fedora-36 root host
  • PR System tests: fix three races #15794
    • 09-14 14:10
• fedora-36 : sys podman fedora-36 rootless host
  • PR podman pod logs -l no longer panics #15846
    • 09-29 10:32
  • PR label.Relabel third option is shared not recurse #15844
    • 09-17 08:32
  • PR fix ci: update systemd generate unit test #15796
    • 09-14 09:34
  • PR System tests: cleanup in --format test #15786
    • 09-14 13:49
  • PR Add support for podman context as alias to podman system connection #15072
    • 09-15 09:34
• fedora-36 : sys podman fedora-36 rootless host
  • PR auto-update: validate container image #15933
    • 09-26 12:53
• fedora-36-aarch64 : sys podman fedora-36-aarch64 root host
  • PR Proof of concept: nightly dependency treadmill #15910
    • 09-24 09:08
• fedora-36 : sys podman fedora-36 rootless host
  • PR Ensure that the DF endpoint updated volume refcount #15753
    • 09-12 22:10
• fedora-36 : sys podman fedora-36 root host
  • PR Update buildah and c/common to latest #15695
    • 09-09 07:22

Looks like Sept 9 is the first logged instance.

So far, f36 only (both amd64 and aarch64), root and rootless.

[vrothberg]

Will take a look, thanks!

[edsantiago]

Possibly related to #16062?

[vrothberg]

I recently attempted to fix some issues down there. @edsantiago, do you know when the flake happened for the first time?

[vrothberg]

I recently attempted to fix some issues down there. @edsantiago, do you know when the flake happened for the first time?

Just saw it at the bottom "Looks like Sept 9 is the first logged instance."

[vrothberg]

I found one potential race that #16118 addresses. The potential race looks extreeeemely narrow to me, so I am not sure whether the PR fixes this issue.

Let's keep an eye open and please let me know if the flakes pops up again after #16118 merges.

[vrothberg]

Closing as #16118 merged. I am not 100 percent sure it fixes the flake but it's the only potential source for the flake I could spot so far. Let's reopen in case it continues.