Allow health indicators to run with a specified timeout

[philwebb]

Originally discussed in #4391 for datasource timeouts, but the problem is more broadly applicable to all health indicators.

[gf-smtzgr]

This would be neat.
You might also want to separate the status computation from the status retrieval in addition to timeouts. Imho the retrieval of a system status should not be affected by the system status, i.e. if database answers are slow that should not mean the health check should be slow. Thus, ideally a separate thread (pool) regularly updates the health status such that the request handling just needs to readout the current status flags rather than do the status checks to answer the request.

[wilkinsona]

Thus, ideally a separate thread (pool) regularly updates the health status such that the request handling just needs to readout the current status flags rather than do the status checks to answer the request.

That's #25459.

[gf-smtzgr]

@wilkinsona thanks! then I can follow that, too.