Configure the header list size in jetty client to more than 8Kb

[Niteshkumar152]

jetty Version=11.0.20
Java Version = 17

I am using the http2 Client of Jetty(plain text).
Following is the code snipped of initializing the HTTP client:

public void init(){

     SslContextFactory sslContextFactory = new SslContextFactory.Client(true);
     ClientConnector clientConnector= new ClientConnector() {
         protected void configure(SelectableChannel selectable) throws IOException {
            super.configure(selectable);
	    if (selectable instanceof NetworkChannel) {
			NetworkChannel channel = (NetworkChannel)selectable;
			channel.setOption(java.net.StandardSocketOptions.SO_KEEPALIVE,
					tcpConfigOptionProvider.getTcpKeepalive().getEnable());
                        // Set keepalive parameters only if it is enabled
					if(tcpConfigOptionProvider.getTcpKeepalive().getEnable()) {
						channel.setOption(jdk.net.ExtendedSocketOptions.TCP_KEEPIDLE,
								Integer.parseInt(StringUtils.chop(tcpConfigOptionProvider.getTcpKeepalive().getTime())));
						channel.setOption(jdk.net.ExtendedSocketOptions.TCP_KEEPINTERVAL,
								Integer.parseInt(StringUtils.chop(tcpConfigOptionProvider.getTcpKeepalive().getInterval())));
						channel.setOption(jdk.net.ExtendedSocketOptions.TCP_KEEPCOUNT,
								tcpConfigOptionProvider.getTcpKeepalive().getProbes());
					}
                                        tcpKeepaliveChannelCofigDetails(channel);

				}
			}
		};
		
		clientConnector.setSslContextFactory((SslContextFactory.Client) sslContextFactory);

		HTTP2Client http2Client = new HTTP2Client(clientConnector);
		http2Client.setMaxConcurrentPushedStreams(maxConcurrentPushedStreams);

		HttpClientTransportOverHTTP2 transport = new HttpClientTransportOverHTTP2(http2Client);
		transport.setUseALPN(false);
		this.httpClient = new HttpClient(transport);
		httpClient.addBean(new JettyConnectionMetrics(SoothsayerMetrics.getInstance().getRegistry()));
		this.httpClient.setIdleTimeout(idleTimeout);
		this.httpClient.setConnectTimeout(httpClientConnectionTimeout);

		this.httpClient.setMaxRequestsQueuedPerDestination(maxRequestsQueuedPerDestination);
		this.httpClient.setMaxConnectionsPerDestination(maxConnectionsPerDestination);
		this.httpClient.setFollowRedirects(false);
		this.httpClient.setRemoveIdleDestinations(true);
		//httpClient.getRequestListeners().add(new JettyClientRequestMetrics());
		httpClient.addBean(new ConnectionStatistics());
		httpClient.addBean(new JettyConnectionMetrics(SoothsayerMetrics.getInstance().getRegistry()));
		this.httpClient.setUserAgentField(null);

		logger.info("Jetty Max Requests: configured value: {}, set value: {}", maxRequestsQueuedPerDestination,
				this.httpClient.getMaxRequestsQueuedPerDestination());
		logger.info("Jetty Idle Timeout: configured value: {}, set value: {}", idleTimeout,
				this.httpClient.getIdleTimeout());
		logger.info("Jetty Max Connections Per Destination: configured value: {}, set value: {}",
				maxConnectionsPerDestination, this.httpClient.getMaxConnectionsPerDestination());
		logger.info("Jetty Http Client Connection Timeout: configured value: {}, set value: {}",
				httpClientConnectionTimeout, this.httpClient.getConnectTimeout());
		try {
			httpClient.start();
			httpClient.getContentDecoderFactories().clear();
		} catch (Exception e) {
			
		}

		ClientHttpConnector httpConnector = new JettyClientHttpConnector(httpClient);
		this.httpDestWebClient = WebClient.builder().codecs(configurer -> configurer.defaultCodecs()
						.maxInMemorySize(2048))
				.clientConnector(httpConnector).baseUrl(extUrl).build();
		
		}

I want to configure jetty client such that I can send headers greater than 8Kb in the request. Currently I believe this parameter is responsible for configuring the header size limit MAX_HEADER_LIST_SIZE.

Please suggest how I can configure MAX_HEADER_LIST_SIZE the way I am creating the client.

Thanks in Advance!

[joakime]

What header are you wanting to send at over 8kb?

If it's the Cookie header, know that the Cookie spec is changing to follow industry best practice (on HTTP/2 and HTTP/3) of sending multiple Cookie headers instead of one.

[Niteshkumar152]

Hi @joakime
Thanks for replying.

No, i want to send a custom header whose size will go beyond 8kb.

[joakime]

No, i want to send a custom header whose size will go beyond 8kb.

I feel that would be very unfriendly to HPACK.

[Niteshkumar152]

@joakime Yes, I agree with it.

[Niteshkumar152]

Currently when I tried sending the header beyond 8Kb, I saw exceptions related to the same.

So, I wanted to know if I can configure the jetty client such that the header can be sent without issues.

[ekupcik]

What header are you wanting to send at over 8kb?

If it's the Cookie header, know that the Cookie spec is changing to follow industry best practice (on HTTP/2 and HTTP/3) of sending multiple Cookie headers instead of one.

Just as a remark: Kerberos/SPNEGO headers can get larger than that if the user is in many groups etc.

[joakime]

Just as a remark: Kerberos/SPNEGO headers can get larger than that if the user is in many groups etc.

Yup, and Microsoft specifically calls out Windows authentication (NTLM/Kerberos/Negotiate) as not supported on HTTP/2 due to HPACK performance issues that those large headers cause.

[Niteshkumar152]

Also, it would be great if you could tell me how can I configure the header size?

[joakime]

The header size limits on HTTP/2 is determined by what the remote can handle.
By default, per spec, MAX_HEADERS_LIST_SIZE is infinite, and HEADER_TABLE_SIZE is 4096.

See https://datatracker.ietf.org/doc/html/rfc7540#section-11.3

The HEADER_TABLE_SIZE can be configured in Http2Client with both setMaxDecoderTableCapacity() and getMaxEncoderTableCapacity().

It is not recommended to use headers of this size, as it negates too many of the benefits of HTTP/2 (you will likely have worse performance on HTTP/2 than on HTTP/1 with headers of this size, something even grpc has discovered).
Those huge headers will impact all streams on the same connection, if just one header update fails, the entire connection is torn down and all streams along with it.

Know that most HTTP/2 server implementations do not support headers over 4k total (as in all headers, not just a single one).

Since you didn't share the logs or stacktrace or errors you are getting, we don't know where the problem is happening (Jetty client side, server error, http2 protocol error, WebClient error, etc).
The changes you are wanting to do can very likely have zero impact on your results, as you might not be in control over the limits being imposed on you.

[Niteshkumar152]

Hi @joakime

This the exception that I was referring to earlier, please provide your inputs on it.
Thanks

"message":"Failure generating HeadersFrame@990c648#1[end=false,{POST{u=http:// localhost:9999/testService/v1/custom/endPoint1,HTTP/2.0,h=22,cl=201,p=null}},priority=null]","thrown":{"message":"Header size 17457 > 8192","name":"org.eclipse.jetty.http2.hpack.HpackException.SessionException","e xtendedStackTrace":[{"class":"org.eclipse.jetty.http2.hpack.HpackEncoder","method":"encode","file":"HpackEncoder.java", "line":289},{"class":"org.eclipse.jetty.http2.generator.FrameGenerator","method":"encode","file":"FrameGenerator.java", "line":56},{"class":"org.eclipse.jetty.http2.generator.HeadersGenerator","method":"generateHeaders","file":"HeadersGene rator.java","line":65},{"class":"org.eclipse.jetty.http2.generator.HeadersGenerator","method":"generate","file":"Header sGenerator.java","line":52},{"class":"org.eclipse.jetty.http2.generator.Generator","method":"control","file":"Generator .java","line":105},{"class":"org.eclipse.jetty.http2.HTTP2Session$ControlEntry","method":"generate","file":"HTTP2Sessio n.java","line":1289},{"class":"org.eclipse.jetty.http2.HTTP2Flusher","method":"process","file":"HTTP2Flusher.java","lin e":215},{"class":"org.eclipse.jetty.util.IteratingCallback","method":"processing","file":"IteratingCallback.java","line ":243},{"class":"org.eclipse.jetty.util.IteratingCallback","method":"iterate","file":"IteratingCallback.java","line":22 4},{"class":"org.eclipse.jetty.http2.HTTP2Session$StreamsState","method":"flush","file":"HTTP2Session.java","line":2387 },{"class":"org.eclipse.jetty.http2.HTTP2Session$StreamsState

[joakime]

Seems that you received a HTTP/2 Settings frame that said the server can only support size 8192.
The local side rejected your attempt at using a larger size as it violated the HPACK settings on that stream.

Also, Jetty 11 is at End of Community Support.

• EoCS (End of Community Support) for Jetty 10 / Jetty 11 - January 2024 #10485

You should be using Jetty 12 at this point in time.

[Niteshkumar152]

Sure @joakime
Thanks for your inputs !!

[Niteshkumar152]

After configuring the remote server to allow more than 8Kb header size, I no longer see the exception.

I was wondering what is the max header size jetty client can support when it receives big headers in response from server.
Also, is there any way to configure the same.

Thanks!!

[sbordet]

@Niteshkumar152 on the client you want to configure the max response header size with HTTP2Client.setMaxResponseHeadersSize().

Closing this issue as answered, please report back if it worked for you on the client too.