Authenticated tag is wrong on streams

[mikerabat]

The routine EncodeStream creates a different authentication tag as the EncodeBytes if the field
DatatToAuthenticate is used.

The problems seems to be that EncodeStream splits the buffer in 8kB blocks and calls EncodeGCM multiple
times. Here the DataToAuthenticate field is used on every call which means that it is mengled into the
tag calculatation on each iteration.

EncodeBytes calls the EncodeGCM routine only once on the whole buffer. Check out the attached project (unfortunately I could not upload a dpr file only a txt file...)

FailStreamEncrypt.txt

On a different matter... would it be possible to add an additional parameter to the EncodeStream function that allows
to change the endianess of the stream? There are built in classes that allow to operate on TBytes but not on streams.
(Or... did I miss something?)
My background here is to create an encrypted firmeware file for a microcontroller which uses an ARM CPU

[MHumm]

Lets fix the bug first before discussing further functionality improvements. Do you suppose to pass the DataToAuthenticate in EncodeStream only on the last EncodeGCM call? Would that fix this bug?

[mikerabat]

Hi!I think so. There should not be a differnce between a stream and a full block right? Maybe I‘ll try to tackle the issue in conjunction with the other one…  (if you like) I never tried to create a pull request but this could be a nice possibility :)Von meinem iPhone gesendetAm 22.04.2025 um 19:45 schrieb Markus ***@***.***>: Lets fix the bug first before discussing further functionality improvements. Do you suppose to pass the DataToAuthenticate in EncodeStream only on the last EncodeGCM call? Would that fix this bug?—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***> MHumm left a comment (MHumm/DelphiEncryptionCompendium#87) Lets fix the bug first before discussing further functionality improvements. Do you suppose to pass the DataToAuthenticate in EncodeStream only on the last EncodeGCM call? Would that fix this bug? —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>

[MHumm]

Feel free to provide a suggested fix. I'd add you to notice.txt as contributor ;-)
If it shouldn't work as pull request for you we'll find an alternative way. Just let me know here if that's the case.

[mikerabat]

I was not able to commit anything to development - however I attached the files in "Source" and one file in the Unit test folder that
should cover tests for bugs #86 and #87.
Let me know if that helps.

This is for block wise encoding only - there has been no work on the decoding part too. In addition it lacks some comments...

Source.zip

TestDECCipherModesGCM.zip

[MHumm]

You are not able to commit, because you do not yet have commit permissions to this repo. The other way most folks prefere would be to create a fork, linkt that with this repo and then you can commit to your fork and create a pull request from that one.

I'll try to have a look at your code within the next few days.
Btw. may I close #86 now as fixed?

[mikerabat]

Yes... #86 looks fixed now :)

I have an update that does the decoding part as well. Unfortunately the previous encode/decode stream tests do not work any more... I have no idea yet why, maybe you can find something,

Source.zip

[MHumm]

The update for decoding is for #87, right?

[mikerabat]

Yes - the thing is the original stream unit tests fail now... Maybe the tests are based on the "original" assumption on how the streaming should work... I don't know. Can you take a look at it?

[MHumm]

We need to check which test data is being used. I'm not sure at the moment whether DEC has "official" test data, means from a standard body, for GCM. If it has/unses that the stream and non stream tests should provide the same results.

[MHumm]

Can you write a small test application comparing the use of some of the test data from the already implemented "non stream" unit tests and feed this to both: non stream methods of DEC and stream methods and compare whether that delivers the same results? I'm quite busy at the moment and will thus not find the time during the next few weeks to look at it. The comparison should show if we really have a bug in the stream handling of this algorithm and you can also implement as 3rd variant your fixed stream variant. This should provide the same results than the non stream code.

[mikerabat]

The last zip file I provided actually implements the following test:

• Create a buffer with 32k of bytes
• Define a small header that is used for the DataToAuthenticate property
• One time the buffer is fed to "EncodeBytes" and the second time to "EncodeStream"
• The Auth tags are then compared
• In addition the stream is decoded and checked with the updated GCM method

This test seems to work but the older stream tests fail now :/

[mikerabat]

Looking at the streaming tests - especially in the GCM section - they partly do not make a lot of sense to me...
The internal Encode/decode stream functions utilize a chunking scheme of 8kB per Block. From my perspective these
were ment to be called only once right?
The actuall stream testing function run "chunking" themself by multiple calls on Encode/Decode Stream with a chunk size.
This creates the same problem in the GCM as the initial described problem - multiple calls to Encode/DecodeStream include
multiple calculation on the AuthenticatedData item (aka on each call). So my question now is:

• Is the library meant to be used that way? Actually having multiple EncodeBytes/EncodeStream calls in one "session"
• If so should - in the GCM case - one start such a session via a separate calls? (Begin End...)
• The current implementation does not allow this nevertheless the tests are actually pointing in that direction....