Triple protocol http1 upgrade support

[walklown]

What is the purpose of the change

Triple protocol http1 upgrade support. For issue [Feature][3.3] Triple protocol http1 upgrade support

Brief changelog

Add Http2ServerUpgradeCodec to http1 channel handlers.

Verifying this change

https://github.com/walklown/dubbo-pr-test/tree/main/pr-test-triple

[AlbumenJ]

@oxsean PTAL

[oxsean]

Test org.apache.dubbo.demo.provider.ApiProvider with curl:

curl.exe -v --http2 -k -H "Content-Type: application/json" -d '"asd"' http://127.0.0.1:50051/org.apache.dubbo.demo.GreeterService/sayHelloAsync

but upgrade failed, could you please take a look?

[oxsean]

Please look into whether Application-Layer Protocol Negotiation is supported in addition to h2c:
https://matthewparrilla.com/post/negotiation-http2-alpn-tls-handshake/
https://imququ.com/post/protocol-negotiation-in-http2.html

[walklown]

Test org.apache.dubbo.demo.provider.ApiProvider with curl:

curl.exe -v --http2 -k -H "Content-Type: application/json" -d '"asd"' http://127.0.0.1:50051/org.apache.dubbo.demo.GreeterService/sayHelloAsync

but upgrade failed, could you please take a look?

Because you specified a http scheme (and not https), and asked curl to use HTTP/2, then curl will attempt to perform a HTTP/1.1 upgrade to HTTP/2, as can be seen from the logs.

Typical HTTP/1.1 upgrades are performed using GET, not POST, notably the HTTP/1.1 upgrade to WebSocket.

The server does not seem to be prepared to accept a POST with a body as an attempt to upgrade and replies with 413 because it does not expect a body.

If you try a GET without body it will likely succeed.

Alternatively, if you know that port 8889 accepts prior-knowledge clear-text HTTP/2 (that is, you can send directly HTTP/2 bytes to that port without having to perform a HTTP/1.1 upgrade), you can try:

curl -v --http2-prior-knowledge -X POST -k -H "content-type:application/json" -d '["myFirstParameter"]' http://127.0.0.1:50052/org.apache.dubbo.springboot.demo.DemoService/sayHello

`$ curl -v --http2-prior-knowledge -X POST -k -H "content-type:application/json" -d '["myFirstParameter"]' http://127.0.0.1:50052/org.apache.dubbo.springboot.demo.DemoService/sayHello
Note: Unnecessary use of -X or --request, POST is already inferred.

• Trying 127.0.0.1:50052...
• Connected to 127.0.0.1 (127.0.0.1) port 50052
• [HTTP/2] [1] OPENED stream for http://127.0.0.1:50052/org.apache.dubbo.springboot.demo.DemoService/sayHello
• [HTTP/2] [1] [:method: POST]
• [HTTP/2] [1] [:scheme: http]
• [HTTP/2] [1] [:authority: 127.0.0.1:50052]
• [HTTP/2] [1] [:path: /org.apache.dubbo.springboot.demo.DemoService/sayHello]
• [HTTP/2] [1] [user-agent: curl/8.5.0]
• [HTTP/2] [1] [accept: /]
• [HTTP/2] [1] [content-type: application/json]
• [HTTP/2] [1] [content-length: 20]

POST /org.apache.dubbo.springboot.demo.DemoService/sayHello HTTP/2
Host: 127.0.0.1:50052
User-Agent: curl/8.5.0
Accept: /
content-type:application/json
Content-Length: 20

< HTTP/2 200
< te: trailers
< content-type: application/json
<

• Connection #0 to host 127.0.0.1 left intact
  "Hello myFirstParameter"`

[oxsean]

Test org.apache.dubbo.demo.provider.ApiProvider with curl:

curl.exe -v --http2 -k -H "Content-Type: application/json" -d '"asd"' http://127.0.0.1:50051/org.apache.dubbo.demo.GreeterService/sayHelloAsync

but upgrade failed, could you please take a look?

Because you specified a http scheme (and not https), and asked curl to use HTTP/2, then curl will attempt to perform a HTTP/1.1 upgrade to HTTP/2, as can be seen from the logs.

Typical HTTP/1.1 upgrades are performed using GET, not POST, notably the HTTP/1.1 upgrade to WebSocket.

The server does not seem to be prepared to accept a POST with a body as an attempt to upgrade and replies with 413 because it does not expect a body.

If you try a GET without body it will likely succeed.

Alternatively, if you know that port 8889 accepts prior-knowledge clear-text HTTP/2 (that is, you can send directly HTTP/2 bytes to that port without having to perform a HTTP/1.1 upgrade), you can try:

curl -v --http2-prior-knowledge -X POST -k -H "content-type:application/json" -d '["myFirstParameter"]' http://127.0.0.1:50052/org.apache.dubbo.springboot.demo.DemoService/sayHello

`$ curl -v --http2-prior-knowledge -X POST -k -H "content-type:application/json" -d '["myFirstParameter"]' http://127.0.0.1:50052/org.apache.dubbo.springboot.demo.DemoService/sayHello Note: Unnecessary use of -X or --request, POST is already inferred.

• Trying 127.0.0.1:50052...
• Connected to 127.0.0.1 (127.0.0.1) port 50052
• [HTTP/2] [1] OPENED stream for http://127.0.0.1:50052/org.apache.dubbo.springboot.demo.DemoService/sayHello
• [HTTP/2] [1] [:method: POST]
• [HTTP/2] [1] [:scheme: http]
• [HTTP/2] [1] [:authority: 127.0.0.1:50052]
• [HTTP/2] [1] [:path: /org.apache.dubbo.springboot.demo.DemoService/sayHello]
• [HTTP/2] [1] [user-agent: curl/8.5.0]
• [HTTP/2] [1] [accept: /]
• [HTTP/2] [1] [content-type: application/json]
• [HTTP/2] [1] [content-length: 20]

POST /org.apache.dubbo.springboot.demo.DemoService/sayHello HTTP/2
Host: 127.0.0.1:50052
User-Agent: curl/8.5.0
Accept: /
content-type:application/json
Content-Length: 20

< HTTP/2 200 < te: trailers < content-type: application/json <

• Connection #0 to host 127.0.0.1 left intact
  "Hello myFirstParameter"`

Test org.apache.dubbo.demo.provider.ApiProvider with curl:

curl.exe -v --http2 -k -H "Content-Type: application/json" -d '"asd"' http://127.0.0.1:50051/org.apache.dubbo.demo.GreeterService/sayHelloAsync

but upgrade failed, could you please take a look?

Because you specified a http scheme (and not https), and asked curl to use HTTP/2, then curl will attempt to perform a HTTP/1.1 upgrade to HTTP/2, as can be seen from the logs.

Typical HTTP/1.1 upgrades are performed using GET, not POST, notably the HTTP/1.1 upgrade to WebSocket.

The server does not seem to be prepared to accept a POST with a body as an attempt to upgrade and replies with 413 because it does not expect a body.

If you try a GET without body it will likely succeed.

Alternatively, if you know that port 8889 accepts prior-knowledge clear-text HTTP/2 (that is, you can send directly HTTP/2 bytes to that port without having to perform a HTTP/1.1 upgrade), you can try:

curl -v --http2-prior-knowledge -X POST -k -H "content-type:application/json" -d '["myFirstParameter"]' http://127.0.0.1:50052/org.apache.dubbo.springboot.demo.DemoService/sayHello

`$ curl -v --http2-prior-knowledge -X POST -k -H "content-type:application/json" -d '["myFirstParameter"]' http://127.0.0.1:50052/org.apache.dubbo.springboot.demo.DemoService/sayHello Note: Unnecessary use of -X or --request, POST is already inferred.

• Trying 127.0.0.1:50052...
• Connected to 127.0.0.1 (127.0.0.1) port 50052
• [HTTP/2] [1] OPENED stream for http://127.0.0.1:50052/org.apache.dubbo.springboot.demo.DemoService/sayHello
• [HTTP/2] [1] [:method: POST]
• [HTTP/2] [1] [:scheme: http]
• [HTTP/2] [1] [:authority: 127.0.0.1:50052]
• [HTTP/2] [1] [:path: /org.apache.dubbo.springboot.demo.DemoService/sayHello]
• [HTTP/2] [1] [user-agent: curl/8.5.0]
• [HTTP/2] [1] [accept: /]
• [HTTP/2] [1] [content-type: application/json]
• [HTTP/2] [1] [content-length: 20]

POST /org.apache.dubbo.springboot.demo.DemoService/sayHello HTTP/2
Host: 127.0.0.1:50052
User-Agent: curl/8.5.0
Accept: /
content-type:application/json
Content-Length: 20

< HTTP/2 200 < te: trailers < content-type: application/json <

• Connection #0 to host 127.0.0.1 left intact
  "Hello myFirstParameter"`

There is no documentation mentioned that methods other than GET are not supported. I think it might be because you haven't set maxContentLength
https://github.com/netty/netty/blob/4379f0b92dc1ae375bba9afa1c279b5aa867a3eb/codec-http/src/main/java/io/netty/handler/codec/http/HttpServerUpgradeHandler.java#L190

--http2-prior-knowledge will skip negotiation, therefore this test is meaningless.

[walklown]

Please look into whether Application-Layer Protocol Negotiation is supported in addition to h2c: https://matthewparrilla.com/post/negotiation-http2-alpn-tls-handshake/ https://imququ.com/post/protocol-negotiation-in-http2.html

Please look into whether Application-Layer Protocol Negotiation is supported in addition to h2c: https://matthewparrilla.com/post/negotiation-http2-alpn-tls-handshake/ https://imququ.com/post/protocol-negotiation-in-http2.html

The current common implementation of ALPN is through extended TLS. TLS processing in Dubbo is in ‘org.apache.dubbo.remoting.transport.netty4.NettyPortUnificationServerHandler’, before confirming the RPC protocol. Extensions to this implementation will have an impact on all RPC protocols. Could you please confirm whether support is needed at this level?

[oxsean]

Please look into whether Application-Layer Protocol Negotiation is supported in addition to h2c: https://matthewparrilla.com/post/negotiation-http2-alpn-tls-handshake/ https://imququ.com/post/protocol-negotiation-in-http2.html

Please look into whether Application-Layer Protocol Negotiation is supported in addition to h2c: https://matthewparrilla.com/post/negotiation-http2-alpn-tls-handshake/ https://imququ.com/post/protocol-negotiation-in-http2.html

The current common implementation of ALPN is through extended TLS. TLS processing in Dubbo is in ‘org.apache.dubbo.remoting.transport.netty4.NettyPortUnificationServerHandler’, before confirming the RPC protocol. Extensions to this implementation will have an impact on all RPC protocols. Could you please confirm whether support is needed at this level?

I think so, but we need to confirm whether adding ALPN support cause a regression in protocols other than Triple, such as the Dubbo protocol.

[walklown]

There is no documentation mentioned that methods other than GET are not supported. I think it might be because you haven't set maxContentLength https://github.com/netty/netty/blob/4379f0b92dc1ae375bba9afa1c279b5aa867a3eb/codec-http/src/main/java/io/netty/handler/codec/http/HttpServerUpgradeHandler.java#L190

--http2-prior-knowledge will skip negotiation, therefore this test is meaningless.

You are right, thanks for the guidance. I have submitted a fix

[walklown]

Please look into whether Application-Layer Protocol Negotiation is supported in addition to h2c: https://matthewparrilla.com/post/negotiation-http2-alpn-tls-handshake/ https://imququ.com/post/protocol-negotiation-in-http2.html

The current common implementation of ALPN is through extended TLS. TLS processing in Dubbo is in ‘org.apache.dubbo.remoting.transport.netty4.NettyPortUnificationServerHandler’, before confirming the RPC protocol. Extensions to this implementation will have an impact on all RPC protocols. Could you please confirm whether support is needed at this level?

I think so, but we need to confirm whether adding ALPN support cause a regression in protocols other than Triple, such as the Dubbo protocol.

OK, I will submit a commit try to implement it.

[walklown]

I think so, but we need to confirm whether adding ALPN support cause a regression in protocols other than Triple, such as the Dubbo protocol.

Submitted a commit. Please take a look, thanks.

My test:
https://github.com/walklown/dubbo-pr-test/blob/main/pr-test-triple/dubbo-triple-provider/src/main/java/org/apache/dubbo/springboot/demo/provider/ProviderApplication.java
https://github.com/walklown/dubbo-pr-test/blob/main/pr-test-triple/dubbo-triple-netty-http1-upgrade/src/main/java/com/walklown/prtest/triple/netty/Http2ALPNClient.java

[oxsean]

LGTM, but have you tested no impact to dubbo protocol?
And It would be good to place your test as a unit test in the Dubbo project, or you could put it in the integration tests as well:
https://github.com/apache/dubbo-integration-cases

[walklown]

LGTM, but have you tested no impact to dubbo protocol? And It would be good to place your test as a unit test in the Dubbo project, or you could put it in the integration tests as well: https://github.com/apache/dubbo-integration-cases

Submitted some test cases to apache/dubbo-integration-cases#22

1. Http2ClientT: Test on triple protocol to support other clients http1 to upgrade http2
2. DubboConsumerT: triple protocol and dubbo protocol RPC function testing, including turning on and off ssl

[walklown]

@AlbumenJ The problem has been solved, PTAL.

[oxsean]

@walklown Sorry, I forgot commit the comments, please take a look.

[walklown]

@walklown Sorry, I forgot commit the comments, please take a look.

Submitted, thanks for your guidance !

[walklown]

The check item 'dubbo-samples-test-13787' failed because the tests attribute in 'case-configuration.yml' was set incorrectly. I think it should be replaced with

 tests:
   - "**/*IT.class"

@AlbumenJ

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 38.52%. Comparing base (a12975a) to head (8238e49).
Report is 490 commits behind head on 3.3.

❗ Current head 8238e49 differs from pull request most recent head 3836719

Please upload reports for the commit 3836719 to get more accurate results.

Additional details and impacted files

@@            Coverage Diff             @@
##              3.3   #14026      +/-   ##
==========================================
- Coverage   38.55%   38.52%   -0.04%     
==========================================
  Files        1895     1896       +1     
  Lines       79272    79313      +41     
  Branches    11528    11529       +1     
==========================================
- Hits        30560    30552       -8     
- Misses      44439    44483      +44     
- Partials     4273     4278       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.