Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance header name validation exception message #2924

Merged
merged 6 commits into from
May 21, 2024

Conversation

idelpivnitskiy
Copy link
Member

@idelpivnitskiy idelpivnitskiy commented May 13, 2024

Motivation:

Provide more details about a illegal header name.

Modifications:

  • Use TokenValidatingProcessor that captures more context for generating an exception message, if necessary;
  • Add benchmark to measure performance before/after change;

Result:

Users can see what header name has an illegal character as well as the position (index) of an illegal character.


This approach captures more context and doesn't use another try-catch with wrapping, but allocates a new lambda for every token validation.

When users add/set illegal header name:

before

Exception in thread "main" io.servicetalk.utils.internal.IllegalCharacterException: '�' (0x00), expected [! / # / $ / % / & / ' / * / + / - / . / ^ / _ / ` / | / ~ / DIGIT / ALPHA]
	at io.servicetalk.http.api.HeaderUtils.validateTokenChar(HeaderUtils.java:831)
	at io.servicetalk.buffer.api.CharSequences.forEachByte(CharSequences.java:126)
	at io.servicetalk.http.api.HeaderUtils.validateToken(HeaderUtils.java:243)
	at io.servicetalk.http.api.DefaultHttpHeaders.validateHeaderName(DefaultHttpHeaders.java:591)
	at io.servicetalk.http.api.DefaultHttpHeaders.validateKey(DefaultHttpHeaders.java:572)
	at io.servicetalk.http.api.DefaultHttpHeaders.validateKey(DefaultHttpHeaders.java:45)
	at io.servicetalk.http.api.MultiMap.putExclusive(MultiMap.java:272)
	at io.servicetalk.http.api.DefaultHttpHeaders.set(DefaultHttpHeaders.java:672)
	at io.servicetalk.http.api.HttpMetaData.setHeader(HttpMetaData.java:126)
	at io.servicetalk.http.api.HttpRequestMetaData.setHeader(HttpRequestMetaData.java:486)
	at io.servicetalk.http.api.StreamingHttpRequest.setHeader(StreamingHttpRequest.java:324)
	at io.servicetalk.http.api.DefaultHttpRequest.setHeader(DefaultHttpRequest.java:168)
	at io.servicetalk.examples.http.helloworld.blocking.BlockingHelloWorldClient.main(BlockingHelloWorldClient.java:32)

after

Exception in thread "main" io.servicetalk.utils.internal.IllegalCharacterException: '�' (0x00) at index=7 in header name 'header-�-name', expected [! / # / $ / % / & / ' / * / + / - / . / ^ / _ / ` / | / ~ / DIGIT / ALPHA]
	at io.servicetalk.http.api.HeaderUtils$TokenValidatingProcessor.process(HeaderUtils.java:879)
	at io.servicetalk.buffer.api.CharSequences.forEachByte(CharSequences.java:126)
	at io.servicetalk.http.api.HeaderUtils.validateToken(HeaderUtils.java:246)
	at io.servicetalk.http.api.DefaultHttpHeaders.validateHeaderName(DefaultHttpHeaders.java:591)
	at io.servicetalk.http.api.DefaultHttpHeaders.validateKey(DefaultHttpHeaders.java:572)
	at io.servicetalk.http.api.DefaultHttpHeaders.validateKey(DefaultHttpHeaders.java:45)
	at io.servicetalk.http.api.MultiMap.putExclusive(MultiMap.java:272)
	at io.servicetalk.http.api.DefaultHttpHeaders.set(DefaultHttpHeaders.java:672)
	at io.servicetalk.http.api.HttpMetaData.setHeader(HttpMetaData.java:126)
	at io.servicetalk.http.api.HttpRequestMetaData.setHeader(HttpRequestMetaData.java:486)
	at io.servicetalk.http.api.StreamingHttpRequest.setHeader(StreamingHttpRequest.java:324)
	at io.servicetalk.http.api.DefaultHttpRequest.setHeader(DefaultHttpRequest.java:168)
	at io.servicetalk.examples.http.helloworld.blocking.BlockingHelloWorldClient.main(BlockingHelloWorldClient.java:32)

When an illegal header name is received over network:

before

2024-05-13 13:51:24,884 servicetalk-global-io-executor-1-2 [WARN ] NettyHttpServer$ErrorLoggingHttpSubscriber - [id: 0xeb527ce7, L:/127.0.0.1:8080 - R:/127.0.0.1:51347] Can not decode a message, no more requests will be received on this HTTP/1.1 connection, closing it due to:
io.servicetalk.http.netty.HttpObjectDecoder$StacklessDecoderException: Invalid header name in line 2: header-�-name
Caused by: io.servicetalk.utils.internal.IllegalCharacterException: '�' (0x00), expected [! / # / $ / % / & / ' / * / + / - / . / ^ / _ / ` / | / ~ / DIGIT / ALPHA]
	at io.servicetalk.http.api.HeaderUtils.validateTokenChar(HeaderUtils.java:831) ~[servicetalk-http-api/:?]
	at io.netty.buffer.AbstractByteBuf.forEachByteAsc0(AbstractByteBuf.java:1301) ~[netty-buffer-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.buffer.AbstractByteBuf.forEachByte(AbstractByteBuf.java:1281) ~[netty-buffer-4.1.109.Final.jar:4.1.109.Final]
	at io.servicetalk.buffer.netty.NettyBuffer.forEachByte(NettyBuffer.java:840) ~[servicetalk-buffer-netty/:?]
	at io.servicetalk.buffer.netty.WrappedBuffer.forEachByte(WrappedBuffer.java:784) ~[servicetalk-buffer-netty/:?]
	at io.servicetalk.buffer.api.AsciiBuffer.forEachByte(AsciiBuffer.java:132) ~[servicetalk-buffer-api/:?]
	at io.servicetalk.buffer.api.CharSequences.forEachByte(CharSequences.java:123) ~[servicetalk-buffer-api/:?]
	at io.servicetalk.http.api.HeaderUtils.validateToken(HeaderUtils.java:243) ~[servicetalk-http-api/:?]
	at io.servicetalk.http.api.DefaultHttpHeaders.validateHeaderName(DefaultHttpHeaders.java:591) ~[servicetalk-http-api/:?]
	at io.servicetalk.http.api.DefaultHttpHeaders.validateKey(DefaultHttpHeaders.java:572) ~[servicetalk-http-api/:?]
	at io.servicetalk.http.api.DefaultHttpHeaders.validateKey(DefaultHttpHeaders.java:45) ~[servicetalk-http-api/:?]
	at io.servicetalk.http.api.MultiMap.put(MultiMap.java:226) ~[servicetalk-http-api/:?]
	at io.servicetalk.http.api.DefaultHttpHeaders.add(DefaultHttpHeaders.java:638) ~[servicetalk-http-api/:?]
	at io.servicetalk.http.netty.HttpObjectDecoder.parseHeaderLine(HttpObjectDecoder.java:687) ~[servicetalk-http-netty/:?]
	at io.servicetalk.http.netty.HttpObjectDecoder.parseAllHeaders(HttpObjectDecoder.java:788) ~[servicetalk-http-netty/:?]
	at io.servicetalk.http.netty.HttpObjectDecoder.readHeaders(HttpObjectDecoder.java:718) ~[servicetalk-http-netty/:?]
	at io.servicetalk.http.netty.HttpObjectDecoder.decode(HttpObjectDecoder.java:299) ~[servicetalk-http-netty/:?]
	at io.servicetalk.transport.netty.internal.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:365) ~[servicetalk-transport-netty-internal/:?]
	at io.servicetalk.transport.netty.internal.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:321) ~[servicetalk-transport-netty-internal/:?]
	at io.servicetalk.transport.netty.internal.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:185) ~[servicetalk-transport-netty-internal/:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) [netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.servicetalk.transport.netty.internal.CopyByteBufHandlerChannelInitializer$CopyByteBufHandler.channelRead(CopyByteBufHandlerChannelInitializer.java:98) [servicetalk-transport-netty-internal/:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) [netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) [netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.kqueue.AbstractKQueueStreamChannel$KQueueStreamUnsafe.readReady(AbstractKQueueStreamChannel.java:544) [netty-transport-classes-kqueue-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.kqueue.AbstractKQueueChannel$AbstractKQueueUnsafe.readReady(AbstractKQueueChannel.java:387) [netty-transport-classes-kqueue-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.kqueue.KQueueEventLoop.processReady(KQueueEventLoop.java:218) [netty-transport-classes-kqueue-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.kqueue.KQueueEventLoop.run(KQueueEventLoop.java:296) [netty-transport-classes-kqueue-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) [netty-common-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.109.Final.jar:4.1.109.Final]
	at java.base/java.lang.Thread.run(Thread.java:829) [?:?]

after

2024-05-13 13:47:33,234 servicetalk-global-io-executor-1-3 [WARN ] NettyHttpServer$ErrorLoggingHttpSubscriber - [id: 0xcb92ece0, L:/127.0.0.1:8080 - R:/127.0.0.1:51203] Can not decode a message, no more requests will be received on this HTTP/1.1 connection, closing it due to:
io.servicetalk.http.netty.HttpObjectDecoder$StacklessDecoderException: Invalid character in line 2: '�' (0x00) at index=7 in header name 'header-�-name', expected [! / # / $ / % / & / ' / * / + / - / . / ^ / _ / ` / | / ~ / DIGIT / ALPHA]Caused by: io.servicetalk.utils.internal.IllegalCharacterException: '�' (0x00) at index=7 in header name 'header-�-name', expected [! / # / $ / % / & / ' / * / + / - / . / ^ / _ / ` / | / ~ / DIGIT / ALPHA]
	at io.servicetalk.http.api.HeaderUtils$TokenValidatingProcessor.process(HeaderUtils.java:879) ~[servicetalk-http-api/:?]
	at io.netty.buffer.AbstractByteBuf.forEachByteAsc0(AbstractByteBuf.java:1301) ~[netty-buffer-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.buffer.AbstractByteBuf.forEachByte(AbstractByteBuf.java:1281) ~[netty-buffer-4.1.109.Final.jar:4.1.109.Final]
	at io.servicetalk.buffer.netty.NettyBuffer.forEachByte(NettyBuffer.java:840) ~[servicetalk-buffer-netty/:?]
	at io.servicetalk.buffer.netty.WrappedBuffer.forEachByte(WrappedBuffer.java:784) ~[servicetalk-buffer-netty/:?]
	at io.servicetalk.buffer.api.AsciiBuffer.forEachByte(AsciiBuffer.java:132) ~[servicetalk-buffer-api/:?]
	at io.servicetalk.buffer.api.CharSequences.forEachByte(CharSequences.java:123) ~[servicetalk-buffer-api/:?]
	at io.servicetalk.http.api.HeaderUtils.validateToken(HeaderUtils.java:246) ~[servicetalk-http-api/:?]
	at io.servicetalk.http.api.DefaultHttpHeaders.validateHeaderName(DefaultHttpHeaders.java:591) ~[servicetalk-http-api/:?]
	at io.servicetalk.http.api.DefaultHttpHeaders.validateKey(DefaultHttpHeaders.java:572) ~[servicetalk-http-api/:?]
	at io.servicetalk.http.api.DefaultHttpHeaders.validateKey(DefaultHttpHeaders.java:45) ~[servicetalk-http-api/:?]
	at io.servicetalk.http.api.MultiMap.put(MultiMap.java:226) ~[servicetalk-http-api/:?]
	at io.servicetalk.http.api.DefaultHttpHeaders.add(DefaultHttpHeaders.java:638) ~[servicetalk-http-api/:?]
	at io.servicetalk.http.netty.HttpObjectDecoder.parseHeaderLine(HttpObjectDecoder.java:687) ~[servicetalk-http-netty/:?]
	at io.servicetalk.http.netty.HttpObjectDecoder.parseAllHeaders(HttpObjectDecoder.java:788) ~[servicetalk-http-netty/:?]
	at io.servicetalk.http.netty.HttpObjectDecoder.readHeaders(HttpObjectDecoder.java:718) ~[servicetalk-http-netty/:?]
	at io.servicetalk.http.netty.HttpObjectDecoder.decode(HttpObjectDecoder.java:299) ~[servicetalk-http-netty/:?]
	at io.servicetalk.transport.netty.internal.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:365) ~[servicetalk-transport-netty-internal/:?]
	at io.servicetalk.transport.netty.internal.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:321) ~[servicetalk-transport-netty-internal/:?]
	at io.servicetalk.transport.netty.internal.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:185) ~[servicetalk-transport-netty-internal/:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) [netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.servicetalk.transport.netty.internal.CopyByteBufHandlerChannelInitializer$CopyByteBufHandler.channelRead(CopyByteBufHandlerChannelInitializer.java:98) [servicetalk-transport-netty-internal/:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) [netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) [netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.kqueue.AbstractKQueueStreamChannel$KQueueStreamUnsafe.readReady(AbstractKQueueStreamChannel.java:544) [netty-transport-classes-kqueue-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.kqueue.AbstractKQueueChannel$AbstractKQueueUnsafe.readReady(AbstractKQueueChannel.java:387) [netty-transport-classes-kqueue-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.kqueue.KQueueEventLoop.processReady(KQueueEventLoop.java:218) [netty-transport-classes-kqueue-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.channel.kqueue.KQueueEventLoop.run(KQueueEventLoop.java:296) [netty-transport-classes-kqueue-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) [netty-common-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.109.Final.jar:4.1.109.Final]
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.109.Final.jar:4.1.109.Final]
	at java.base/java.lang.Thread.run(Thread.java:829) [?:?]

@idelpivnitskiy
Copy link
Member Author

Lmk what approach you like more and I will adjust tests as needed.

@idelpivnitskiy idelpivnitskiy changed the title Enhance header name validation exception message Enhance header name validation exception message (approach 2) May 13, 2024
@bryce-anderson
Copy link
Contributor

Both approaches would be fine with me but if I had to pick this is my preference since it gives the index of the offending char.

Motivation:

Provide more details about a illegal header name.

Modifications:

- Use `TokenValidatingProcessor` that captures more context for
generating an exception message, if necessary;

Result:

Users can see what header name has an illegal character as well as the
position (index) of an illegal character.
* DefaultHttpHeadersNameValidationBenchmark.addHeader 8 false thrpt 5 5024017.665 ± 32830.519 ops/s
* DefaultHttpHeadersNameValidationBenchmark.addHeader 8 true thrpt 5 135011.887 ± 1832.187 ops/s
* DefaultHttpHeadersNameValidationBenchmark.addHeader 16 false thrpt 5 2523554.519 ± 8780.660 ops/s
* DefaultHttpHeadersNameValidationBenchmark.addHeader 16 true thrpt 5 68474.840 ± 1154.751 ops/s
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a benchmark and it seems like performance of the "happy-path" was not affected after switching from a method reference to an inner class with extra state. Likely there will be a bit more GC pressure, but overall seems acceptable.
The invalid header validation is expected to become slower because after changes it generates more descriptive exception message.

Copy link
Contributor

@bryce-anderson bryce-anderson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the improved message a lot. 👍

Comment on lines 907 to 916
private static int stringSize(final int value) {
assert value >= 0;
int size = 0;
int tmp = 1;
while (tmp <= value && tmp <= 1_000_000_000) {
++size;
tmp = (tmp << 3) + (tmp << 1);
}
return size;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At a glance it's not obvious what this is doing. What if you just gave it 6 bytes and let it resize in the exceedingly uncommon situation that we are wrong?

Alternatively, if you really want to count digits, would something like this be more readable?

int size = 1;
int test = 10;
while (test <= value && test  <= 1_000_000_000) {
  size++;
  test *= 10;
}

Copy link
Member Author

@idelpivnitskiy idelpivnitskiy May 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated mehod name, added javadoc, renamed local variables, used multiplication

@idelpivnitskiy idelpivnitskiy changed the title Enhance header name validation exception message (approach 2) Enhance header name validation exception message May 20, 2024
@idelpivnitskiy idelpivnitskiy merged commit 2f50382 into apple:main May 21, 2024
15 checks passed
@idelpivnitskiy idelpivnitskiy deleted the headerName2 branch May 21, 2024 15:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants