Optimize HttpRequestOptions storage

[antonfirsov]

Currently, HttpRequestOptions is backed by a Dictionary<string, object?> that contains only a few items in the vast majority of the use cases which is sub-optimal. Instead we could, back it by an array, or even embed a few key-value pairs, and allocate a dictionary/array only if the embedded limit is exceeded.

This would reduce the memory footprint of Metrics enrichment and also improve many other customer scenarios (eg. in dotnet/extensions) which rely on HttpRequestOptions.

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Currently, HttpRequestOptions is backed by a Dictionary<string, object?> that contains only a few items in the vast majority of the use cases which is sub-optimal. Instead we could, back it by an array, or even embed a few key-value pairs, and allocate a dictionary/array only if the embedded limit is exceeded.

This would reduce the memory footprint of Metrics enrichment and also improve several customer scenarios (eg. in dotnet/extensions) which rely on HttpRequestOptions.

Author:	antonfirsov
Assignees:	-
Labels:	area-System.Net.Http
Milestone:	Future

[gfoidl]

FYI: in dotnet/aspnetcore#31360 something similar was done (maybe that can be mirrored to here)

[antonfirsov]

It looks like their AdaptiveCapacityDictionary is used for stuff like query strings and cookies, I expect HttpRequestOptions to be smaller in most cases. We need to gather usage data on typical HttpRequestOptions scenarios before committing to an implementation technique.

BrowserHttpHandler defines 2 options. Enrichment uses a single key.

@dpk83 do you know how many elements do you store typically in HttpRequestOptions in R9? I found 3 HttpRequestOptionsKey usages, but maybe you use the legacy HttpRequestMessage.Properties more.