testing/synctest,runtime: randomize scheduling to protect against Hyrum's Law

[aclements]

The new synctest package makes time deterministic within a testing bubble. One consequence of this is that the goroutine schedule also becomes more deterministic within the bubble. But it's a fragile determinism that depends on the scheduler implementation. If we leave this alone, Hyrum's Law will lead to tests that depend on the details of the scheduler implementation, making the scheduler more difficult to change in the future. We've successfully defended against this before through intentional randomization, most notably in map iteration and the select poll order. We should do the same for synctest and defend against Hyrum's Law by introducing intentional randomization in the selection of which runnable goroutine to run when inside a bubble. We already do this in -race mode and can probably reuse that same mechanism.

cc @neild , @golang/runtime

[neild]

I'm not sure that we actually need to do anything here.

Synctest only makes scheduling more deterministic in the sense that it (by design) makes time deterministic. An event that happens at time T will always happen before one that happens at time T+1.

When time advances in a synctest bubble, it runs timers using the same mechanism as the regular runtime scheduler (runtime.timers.check). I think this results in exactly as much deterministic as usual, for timers scheduled at the same instant. The only place where determinism differs is when two timers are scheduled for almost, but not quite, the same instant--in that case, the usual runtime will probably execute both at once, while synctest will always execute them in order.

[thepudds]

See also #73876.

[mknyszek]

@aclements We discussed this in the weekly triage meeting, and we tend to agree with @neild. If two goroutines become runnable simultaneously (for example, from two timers firing simultaneously, or close in time), it's still as non-deterministic as before. @neild notes there is some unwanted determinism around selecting on timer channels, and maybe we do something there.

Is there something we're missing? Applying okay-after-rc1 for now.

[glycerine]

I tried to make it more obvious what I'm seeing, in this comment-- #73876 (comment)

It sure does not look (pseudo) random to me.

If it makes sense to someone, please help me understand what I'm missing.

Update: nevermind. I found my bug, it was in my printing of the wrong timer.

[gopherbot]

Change https://go.dev/cl/677276 mentions this issue: runtime: randomize order of timers at the same instant in bubbles