bpo-32243: Use monotonic clock for thread timeouts; more flexibility for very small switch intervals

[embray]

This is the work I did toward fixing bpo-32243 (and might be better broken into separate PRs--we'll see).

This updates @vstinner's patch from bpo-23428, and on top of it adds a fix to PyCOND_TIMEDWAIT that makes small adjustments, if needed, to the timeout (in microseconds) if it's so short that the deadline is always in the past.

My experience in profiling the latter fix (just with some print statements) was that on a slow VM, with setting sys.setswitchinterval(1e-6) on a slow system, this would typically go through the while loop I added to PyCOND_TIMEDWAIT twice (i.e. it would increase the timeout once). On rarer occasions it would go through three times. With sys.setswitchinterval(1e-5) it would sometimes go through the loop twice--usually just once. And for longer switch intervals than that it would always go through the loop only once.

This does all unfortunately come at the cost of an unconditional extra call GETTIME().

https://bugs.python.org/issue32243

[methane]

This doesn't compile on macOS.

In file included from Python/ceval.c:145:
In file included from Python/ceval_gil.h:55:
Python/condvar.h:75:11: error: implicit declaration of function 'pthread_condattr_setclock' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    err = pthread_condattr_setclock(&attr, CLOCK_MONOTONIC);
          ^

[embray]

The use of pthread_condattr_setclock is from @vstinner's original patch. It appears this interface is not supported on macOS. So there should be a configure-time check for it, and I suppose the definition of MONOTONIC dependent on its presence.

[embray]

Rebased in current master (though it was a clean merge) and added a configure-time check for pthread_condattr_setclock.

I'm not sure what is the best way to update the configure script in git though, given that I have some different autotools versions on my system.

[ned-deily]

@embray If you have an autoconf around 2.69, that should be good enough. If necessary, the core dev who ultimately might merge the PR can redo it.

[embray]

I think this should do. There were some other minor differences that were unrelated to the change and easily excluded.

[methane]

monotonic clock is implemented in #11723

[embray]

@methane Please reopen, as I think you've partially misunderstood the purpose of this PR, which is to fix this issue https://bugs.python.org/issue32243 , where the GIL can get stuck if sys.setswitchinterval is very small.

The issue is fixed most effectively by using the monotomic clock, which is why this PR includes it, but that isn't actually the main point of it.

That's great that #11723 finally added the monotonic clock support, so I can just rebase this PR now, and it will make the actual bug fix more readily apparent. AFAICT #11723 on its own does not address this issue.

[methane]

I know you tried to solve different issue. I didn't reject issue32243.
It's up to you that creating new pull request or pushing patch to this pull request.

[vstinner]

I would suggest to write a new pull request written on top of the master branch. I would be interested to read it :-)

[embray]

Seems odd to just close but okay. I can make a new PR since it would be slightly different insofar as not including the clock_monotonic updates themselves.