#6850 Causes Null Reference Exception If Attachment Content Gets Into BotState

[clintwar]

Version

4.22.9.0 Microsoft.Bot.Schema

Describe the bug

Microsoft.Bot.Builder.Azure.Blobs.BlobsStorage maintains a JsonSerializer to read and write the state stored in blob storage.
Attachment.Content has a converter specified. If state is read or written where Attachment.Content is read or written into state since BlobsStorage is generally a singleton across the lifetime of the bot process you can introduce unsafe thread access to the List that backs the JsonConverterCollection on a JsonSerializer by adding and removing the AttachmentMemoryStreamConverter to the shared Serializer.

Under many thread access this can introduce a null entry in the array that backs the list and the list will indicate an incorrect size. Then during serialization generally when it visits the converters collection Newtonsoft is not expecting a null object in the converter collection resulting in a null reference exception.

To Reproduce

Serializer and deserialize Attachment Content across multiple threads.

Expected behavior

A null reference exception should not happen. You should not expect that adding a removing a converter from an unknown serializer source is a safe practice.

Screenshots

See the dump output.

Additional context

0:000> !dumpobj /d 7c5c4ec
Name: Microsoft.Bot.Builder.Azure.Blobs.BlobsStorage
MethodTable: 2249dc9c
EEClass: 2245d2e0
Tracked Type: false
Size: 64(0x40) bytes
File: C:\home\site\wwwroot\Microsoft.Bot.Builder.Azure.Blobs.dll
Fields:
MT Field Offset Type VT Attr Value Name
22673a04 4000002 4 ...on.JsonSerializer 0 instance 07c5c548 _jsonSerializer
226747b0 4000003 8 ...obContainerClient 0 instance 07c687d4 _containerClient
07a1c8e8 4000004 c System.Int32 1 instance 0 _checkForContainerExistence
2249d7a8 4000005 14 ...geTransferOptions 1 instance 07c5c500 _storageTransferOptions

....
0:000> !DumpObj /d 07c5c548
Name: Newtonsoft.Json.JsonSerializer
MethodTable: 22673a04
EEClass: 227435e8
Tracked Type: false
Size: 164(0xa4) bytes
File: C:\home\site\wwwroot\Newtonsoft.Json.dll
Fields:
MT Field Offset Type VT Attr Value Name
22672b3c 4000071 28 System.Int32 1 instance 3 _typeNameHandling
22672b98 4000072 2c System.Int32 1 instance 0 _typeNameAssemblyFormatHandling
22672bf4 4000073 30 System.Int32 1 instance 0 _preserveReferencesHandling
22672c50 4000074 34 System.Int32 1 instance 0 _referenceLoopHandling
22672cac 4000075 38 System.Int32 1 instance 0 _missingMemberHandling
22672d08 4000076 3c System.Int32 1 instance 0 _objectCreationHandling
22672d64 4000077 40 System.Int32 1 instance 0 _nullValueHandling
22672dc0 4000078 44 System.Int32 1 instance 0 _defaultValueHandling
22672e1c 4000079 48 System.Int32 1 instance 0 _constructorHandling
22672e78 400007a 4c System.Int32 1 instance 0 _metadataPropertyHandling
2267b080 400007b 4 ...nverterCollection 0 instance 07ef77d4 _converters

...

0:000> !DumpObj /d 07ef77d4
Name: Newtonsoft.Json.JsonConverterCollection
MethodTable: 2267b080
EEClass: 22749498
Tracked Type: false
Size: 12(0xc) bytes
File: C:\home\site\wwwroot\Newtonsoft.Json.dll
Fields:
MT Field Offset Type VT Attr Value Name
187d20c0 40021fd 4 ...Private.CoreLib]] 0 instance 07ef77e0 items

0:000> !DumpObj /d 07ef77e0
Name: System.Collections.Generic.List`1[[Newtonsoft.Json.JsonConverter, Newtonsoft.Json]]
MethodTable: 22675d78
EEClass: 18a4f4c0
Tracked Type: false
Size: 20(0x14) bytes
File: C:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App\8.0.12\System.Private.CoreLib.dll
Fields:
MT Field Offset Type VT Attr Value Name
1a5c06fc 4002271 4 System.__Canon[] 0 instance 0c15d830 _items
07a1c8e8 4002272 8 System.Int32 1 instance 2 _size
07a1c8e8 4002273 c System.Int32 1 instance 133870 _version
1a5c06fc 4002274 4 System.__Canon[] 0 static dynamic statics NYI s_emptyArray

..
0:000> !DumpArray /d 0c15d830
Name: Newtonsoft.Json.JsonConverter[]
MethodTable: 2267604c
EEClass: 07a19e7c
Size: 28(0x1c) bytes
Array: Rank 1, Number of elements 4, Type CLASS
Element Methodtable: 22675d10
[0] 0bb0b510
[1] null
[2] null
[3] null

List thinks size is 2 but really there is only 1 because of the incorrect thread access to the converters collection by this converter

0:000> !DumpObj /d 0bb0b510
Name: Microsoft.Bot.Schema.Converters.AttachmentMemoryStreamConverter
MethodTable: 320564fc
EEClass: 3219af94
Tracked Type: false
Size: 12(0xc) bytes
File: C:\home\site\wwwroot\Microsoft.Bot.Schema.dll
Fields:
None