Feature Request: Add terminationGracePeriodSeconds field in liveness probe

[gane5hvarma]

/area API
/kind good-first-issue
/kind spec

Describe the feature

The kubernetes pod spec allows us to mention terminationGracePeriodSeconds under livenessProbe for a specific container. This helps restarting a failed container. Current the knative service spec ignores this field.

docs for reference: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#probe-level-terminationgraceperiodseconds

My use case:
The liveness probe for the container fails and the container receives a sigterm. The container code went into a deadlock and doesnt respect gracefull shutdown. Now the container waits for pod terminationGracePeriodSeconds which is set by knative. This is taking around 300 second for the pod to restart. I dont want this to setting because of request timeout, but want the container to restart as soon as liveness probe fails.

[flomedja]

This seems like a duplicate of #15599

[gane5hvarma]

Hi @flomedja this is not a duplicate of #15599. 15599 talks about deleting deployment and it talks about terminationGracePeriodSeconds in pod spec.
This issue is about adding terminationGracePeriodSeconds in livenessProbe. This is need if the container get sigterm, it should restart as soon as possible. The pod wont be terminated.

[flomedja]

My bad. I get it.

[gane5hvarma]

@flomedja what do you think of this feature?

[flomedja]

@skonto @dprotaso Was there a specific reason for ignoring the terminationGracePeriodSeconds at the container level ?

[dprotaso]

So Knative by default sets the terminationGracePeriod to the revision's timeoutseconds. This handles the scenario where a request comes in around the time the pod is being terminated - thus we want to handle that request and not fail.

The kubernetes pod spec allows us to mention terminationGracePeriodSeconds under livenessProbe for a specific container.

I'm reading the K8s docs and I really don't see the motivation for this feature. Do you have a KEP link handy?

The container code went into a deadlock and doesnt respect gracefull shutdown

Which container? the queue proxy or the user's app?

[dprotaso]

Ok - I found the KEP and it explained the use

https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/2238-liveness-probe-grace-period#summary

Yeah I think this makes sense to support.

From the KEP the TerminationGracePeriod in the probe is used for abnormal termination when the liveness probes fail.

The top-level TerminationGracePeriod is for normal shutdown.