Optimize AES library to reduce allocations

[TahirAhmadov]

Is it possible to invoke the native encryptor on Spans? I use the Span-based APIs, but I still get a ton of allocations, and I have a suspicion (without diving in to the code) that it has to do with invoking native stuff. In the screenshot, nearly all of the byte[] allocations are also in encryption. I profiled a hot path which renders a lot of encrypted tokens, and allocations are through the roof, where I was hoping to have near zero. I'm calling aes.TryEncryptCbc.

[stephentoub]

Please share a minimal repro.

What's the bracktrace for those byte[] s?

And what OS? What .NET version?

[TahirAhmadov]

Leaving out unrelated stuff:

		public Span<byte> Encrypt(ReadOnlySpan<byte> plainSpan, Span<byte> destination)
		{
			var aes = this._aesPool.Rent();
			int bodyLen = aes.GetCiphertextLengthCbc(plainSpan.Length, PaddingMode.PKCS7);
			if (destination.Length < bodyLen)
				throw new Exception("Provided buffer wasn't large enough for encrypted bytes.");
			if (!aes.TryEncryptCbc(plainSpan, _staticIV, destination, out int bodyWritten, PaddingMode.PKCS7))
				throw new Exception("Encryption error.");
			this._aesPool.Return(aes);
			return destination[..bodyWritten];
		}

Windows 10 22H2, .NET 10 Release (obviously).

[bartonjs]

4216792 / 98212 is 42. I forget what the object overhead is, but feels like it's in the ballpark for a 16 byte buffer to hold the IV state, assuming you're using Windows with an ephemeral key, and thus that "..." is "BasicSymmetricCipherLiteBCrypt".

All three of those things (well, all 97203 triplets, the other 1009 arrays presumably came from somewhere else) should be rather transitory and should get cleaned up during gen0 GC.

Reducing the allocation any further would require some not-insignificant "plumbing" changes, and potentially no longer sharing code between the stateful and stateless encryption paths. So, it's "possible" to get it down lower, but it isn't (or, at minimum "hasn't been") the top of our areas-for-improvement pile. Is ~128 bytes gen0 per call really being a problem for you, or just a somewhat big-looking number on your profiler?

[TahirAhmadov]

I know these do get cleaned up, but in this particular scenario, all that was generated from only 10 HTTP requests (albeit, pretty heavy ones response wise). I specifically started with collection paused, and collected data only for the 10 requests in question (to isolate the problem the best I can). So, it's a total of 12,770,656 bytes for 10 requests, which makes it 1,277,065 bytes per request - I think it's too much load on GC. Also, it's ~300k allocations, so ~30k allocations per request.

[TahirAhmadov]

I suspect that internally, it allocates a byte[], copies from my span to the byte[], invokes native, then copies from byte[] to my destination span. So it's not only an extra allocation, but also the overhead of the copying.
I guess the allocation can be fixed quickly with ArrayPool, but the copying is a problem nonetheless. Which is why I was hoping the spans can be passed as pointers to native, but this is pure speculation on my part.

[bartonjs]

I suspect that internally, it allocates a byte[], copies from my span to the byte[], invokes native, then copies from byte[] to my destination span.

Just for the IV. The data is passed in without a copy, and the encrypted form is written back without a copy. Whether you encrypt one byte or a gigabyte, a single call to TryEncryptCbc is only going to allocate one 16 byte array (since it's AES, and thus a 16-byte IV), one SafeKeyHandle, and one BasicSymmetricCipherLiteBCrypt.

The IV has to be a copy because we take it as input-only, but Windows takes it as in-out, and write the "next call" IV during the same call. With aforementioned plumbing changes we could eliminate that for the one-shots, but it'll still leave you at 2/3 of what you're seeing without even further plumbing changes.

all that was generated from only 10 HTTP requests

That implies you're doing about 9700 calls to TryEncryptCbc for each request. That's... atypical? 😄

int bodyLen = aes.GetCiphertextLengthCbc

It wouldn't change the allocation results here, but it's weird to call GetCiphertextLength and TryEncrypt, as TryEncrypt itself is just GetCiphertextLength, early return, then do the work. If you want a custom exception you can use TryEncryptCbc without GetCiphertextLengthCbc. If you want CryptographicException skip them both and just use EncryptCbc (which is if-not-TryEncryptCbc-throw). GetCiphertextLengthCbc is mostly for "I don't want do to double-and-retry, so tell me the answer and I'll allocate a big enough array, or call whatever.EnsureCapacity, or whatever")

TryEncryptCbc(plainSpan, _staticIV, ...

Encrypting with a static IV means that the same data (assuming the same key) will encrypt to the same output, which leaks "the data is unchanged" across multiple messages. Maybe it's right for what you're doing, but usually a static IV is an anti-pattern. Again, not relevant to the allocation story, but "someone (might be) is wrong on the Internet..."

[TahirAhmadov]

Forgive my ignorance of the plumbing/objects, isn't it possible to get rid of allocating the SafeKeyHandle and BasicSymmetricCipherLiteBCrypt? What do they do? Regarding the IV, since it's always 16 bytes, why isn't it stack allocated in the TryEncryptCbc and then you have a writable Span?

That implies you're doing about 9700 calls to TryEncryptCbc for each request. That's... atypical? 😄

It is, I admit, it's one of those requests which returns many records and each record has a few fields which need to be encrypted. I was just disappointed that even after I started calling Span-based overloads, it still allocates for what should basically be a math operation underneath. (I used to have byte arrays and MemoryStreams, so it was much worse before - legacy code from pre-Span days.)

it's weird to call GetCiphertextLength and TryEncrypt

Point taken, I'll revise that section.

which leaks "the data is unchanged" across multiple messages.

It is for this use case; however, in other cases I should salt it; I'll tackle that next. I've just been spending some time recently improving memory consumption - both allocations and persisted.