proposal: x/net/http2: deprecate WriteScheduler

[neild]

This issue is part of a project to move x/net/http2 into std: #67810

The http2.WriteScheduler interface type defines a scheduler used to select the order in which data is written to HTTP/2 streams. The Server.NewWriteScheduler configuration setting selects a write scheduler to use for server connections. Client connections do not permit configuring the write scheduler.

The package includes three implementations of this interface: Round robin (the default), random, and priority. The priority write scheduler is known to be buggy and the HTTP/2 stream prioritization scheme it implements is deprecated; see #58804 (comment).

Support for user-defined write schedulers is used seldom, if ever, and substantially constrains the HTTP/2 implementation.

Users should not need to configure the HTTP/2 implementation at this level. Of the current set of scheduler options, there is a clear correct choice: The round robin scheduler outperforms the random scheduler, and the priority scheduler is buggy and we have no plans to fix it. If we come up with a better scheduler, we should just make it the default. If we do discover a reason for users to configure this, we can and should provide a simpler configuration setting than what exists now.

I propose deprecating the write scheduler selection mechanism, consisting of the following types and functions:

• FrameWriteRequest
• NewPriorityWriteScheduler
• NewRandomWriteScheduler
• OpenStreamOptions
• PriorityWriteSchedulerConfig
• Server.NewWriteScheduler
• WriteScheduler

I propose that we add a deprecation notice to these types and functions today, and remove support for them in one year. After removing support, servers will ignore the value of the NewWriteScheduler field.

[pkramme]

I strongly disagree with the plan to deprecate this without providing an alternative. I believe that WriteScheduler support (or something else that allows us to make changes) is a good thing to have and I believe that I can show that it has measurable benefits:

Consider these webpagetest.com benchmarks:

1. x/net/http2 priority
2. x/net/http2 random
3. x/net/http2 round-robin
4. net/http default
5. our internal scheduler

In my tests, the write scheduler makes a huge difference in user performance for this http2 priorization test page. Between the current net/http default and x/net/http2 priority the difference in speed index is over 1 second, and LCP timings are 2.5 seconds better. All other x/net/http2 schedulers are as slow as the net/http default. Our own internal scheduler is better than x/net/http2 priority, for this page.

The difference between x/net/http2 priority (or our implementation) and net/http default means an pretty severe decrease in performance, and must not be dismissed out of hand.

Now benchmarks for an actual webpage, this time tested with a low latency / relatively high bandwidth setup:

1. x/net/http2 priority
2. net/http default
3. our internal scheduler

Again, even under far better network conditions, there is a 300ms difference between x/net/http2 priority and net/http default. Our own scheduler falls somewhere in between. Again, 300ms difference on a speed index of 1.7 seconds is huge, considering that this is archived with zero code changes to the actual page. On this page, rendering also begins almost half a second earlier with x/net/http2 priority.

So, at this point there is data to establish that the performance for endusers is not worse and that the net/http default is not universally better. The tested site was randomly chosen, and I am confident that other sites I could test this on would experience at least equal performance decreases with net/http default.

There is also something else to consider: As far as I know, Go's net/http library (with x/net/http2) is the only "ecosystem" where such a change is possible.

• NGINX's priority system is often not working correctly with no desire from the devs to fix it
• Haproxy doesn't implement priorization
• Apache2 has it, but its not changeable without diving into Apache2 C code (which is not feasible)
• Traefik has priorization from x/net/http2
• I believe that grpc is also implementing some scheduler

With the deprecation of priorization from the HTTP2 spec, the way forward should be to add a new scheduler (and necessary framer support) that implements RFC9218, not to remove the option entirely. That said, the round robin default might be a good default when using Go for APIs, but still that should not remove the option to configure Go's webserver for enduser traffic.

Finally: Cloudflare offers "better HTTP2 priorization" as a product, which would be implementable in Go with write schedulers. They created benchmarks showing its value here. I believe that the ability to implement something like this should stay available to Go devs.