System.IO.EndOfStreamException: Stream ended without reaching end of json content

[ramonsmits]

Describe the bug

Description

Method RavenCustomCheckDataStore.GetStats fails with:

Connection id "0HNACNH3HPSGU", Request id "0HNACNH3HPSGU:000000B1": An unhandled exception was thrown by the application.|System.InvalidOperationException: Cannot parse the 'InternalServerError' response. Content: 
 ---> System.IO.EndOfStreamException: Stream ended without reaching end of json content

Expected behavior

Having a stable database

Actual behavior

Instable database

Versions

6.3.2

Steps to reproduce

Unknown

Relevant log output

2025-02-17 10:51:05.5787|52|Error|Microsoft.AspNetCore.Server.Kestrel|Connection id "0HNACNH3HPSGU", Request id "0HNACNH3HPSGU:000000B1": An unhandled exception was thrown by the application.|System.InvalidOperationException: Cannot parse the 'InternalServerError' response. Content: 
 ---> System.IO.EndOfStreamException: Stream ended without reaching end of json content
   at Sparrow.Json.JsonOperationContext.ParseToMemoryAsync(Stream stream, String documentId, UsageMode mode, MemoryBuffer bytes, IBlittableDocumentModifier modifier, Nullable`1 token, Int32 maxSize) in /_/62017/src/Sparrow/Json/JsonOperationContext.cs:line 805
   at Sparrow.Json.JsonOperationContext.ParseToMemoryAsync(Stream stream, String debugTag, UsageMode mode, IBlittableDocumentModifier modifier, Nullable`1 token) in /_/62017/src/Sparrow/Json/JsonOperationContext.cs:line 559
   at Raven.Client.Exceptions.ExceptionDispatcher.GetJson(JsonOperationContext context, HttpResponseMessage response, Stream stream) in /_/62017/src/Raven.Client/Exceptions/ExceptionDispatcher.cs:line 246
   --- End of inner exception stack trace ---
   at Raven.Client.Exceptions.ExceptionDispatcher.GetJson(JsonOperationContext context, HttpResponseMessage response, Stream stream) in /_/62017/src/Raven.Client/Exceptions/ExceptionDispatcher.cs:line 255
   at Raven.Client.Exceptions.ExceptionDispatcher.Throw(JsonOperationContext context, HttpResponseMessage response, CommandUnsuccessfulResponseBehavior unsuccessfulResponseBehavior) in /_/62017/src/Raven.Client/Exceptions/ExceptionDispatcher.cs:line 90
   at Raven.Client.Exceptions.ExceptionDispatcher.Throw(JsonOperationContext context, HttpResponseMessage response, CommandUnsuccessfulResponseBehavior unsuccessfulResponseBehavior) in /_/62017/src/Raven.Client/Exceptions/ExceptionDispatcher.cs:line 122
   at Raven.Client.Http.Behaviors.DefaultCommandResponseBehavior.TryHandleUnsuccessfulResponseAsync[TResult](JsonOperationContext context, RavenCommand`1 command, HttpResponseMessage response, CommandUnsuccessfulResponseBehavior unsuccessfulResponseBehavior) in /_/62017/src/Raven.Client/Http/Behaviors/DefaultCommandResponseBehavior.cs:line 54
   at Raven.Client.Http.RequestExecutor.HandleUnsuccessfulResponse[TResult](ServerNode chosenNode, Nullable`1 nodeIndex, JsonOperationContext context, RavenCommand`1 command, HttpRequestMessage request, HttpResponseMessage response, String url, SessionInfo sessionInfo, Boolean shouldRetry, CancellationToken token) in /_/62017/src/Raven.Client/Http/RequestExecutor.cs:line 1626
   at Raven.Client.Http.RequestExecutor.ExecuteAsync[TResult](ServerNode chosenNode, Nullable`1 nodeIndex, JsonOperationContext context, RavenCommand`1 command, Boolean shouldRetry, SessionInfo sessionInfo, CancellationToken token) in /_/62017/src/Raven.Client/Http/RequestExecutor.cs:line 1014
   at Raven.Client.Http.RequestExecutor.ExecuteAsync[TResult](ServerNode chosenNode, Nullable`1 nodeIndex, JsonOperationContext context, RavenCommand`1 command, Boolean shouldRetry, SessionInfo sessionInfo, CancellationToken token) in /_/62017/src/Raven.Client/Http/RequestExecutor.cs:line 1044
   at Raven.Client.Documents.Session.AsyncDocumentQuery`1.ExecuteActualQueryAsync(CancellationToken token) in /_/62017/src/Raven.Client/Documents/Session/AsyncDocumentQuery.cs:line 1014
   at Raven.Client.Documents.Session.AsyncDocumentQuery`1.InitAsync(CancellationToken token) in /_/62017/src/Raven.Client/Documents/Session/AsyncDocumentQuery.cs:line 1004
   at Raven.Client.Documents.Session.AsyncDocumentQuery`1.ExecuteQueryOperationInternal(Nullable`1 take, CancellationToken token) in /_/62017/src/Raven.Client/Documents/Session/AsyncDocumentQuery.cs:line 959
   at Raven.Client.Documents.Session.AsyncDocumentQuery`1.ExecuteQueryOperation(Nullable`1 take, CancellationToken token) in /_/62017/src/Raven.Client/Documents/Session/AsyncDocumentQuery.cs:line 941
   at ServiceControl.Persistence.RavenDB.RavenCustomCheckDataStore.GetStats(PagingInfo paging, String status) in /_/src/ServiceControl.Persistence.RavenDB/RavenCustomCheckDataStore.cs:line 55
   at ServiceControl.CustomChecks.CustomCheckController.CustomChecks(PagingInfo pagingInfo, String status) in /_/src/ServiceControl/CustomChecks/Web/CustomCheckController.cs:line 22
   at lambda_method604(Closure, Object)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.AwaitableObjectResultExecutor.Execute(ActionContext actionContext, IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeActionMethodAsync>g__Awaited|12_0(ControllerActionInvoker invoker, ValueTask`1 actionResultValueTask)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeNextActionFilterAsync>g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeInnerFilterAsync>g__Awaited|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
   at Microsoft.AspNetCore.ResponseCompression.ResponseCompressionMiddleware.InvokeCore(HttpContext context)
   at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)

Additional Information

No response

[ramonsmits]

We have reviewed this bug and added it to our bug fix queue. However, there is no ETA for when we will get to this bug as it may be prioritized behind other, more urgent, bugs. We will update this issue when it has a team assigned and work begins.