When validating large CBOR arrays, ItemValidator is prone to stack overflows like this one:
java.lang.StackOverflowError
at java.base/java.lang.invoke.Invokers.checkGenericType(Invokers.java:541)
at scala.runtime.Statics.releaseFence(Statics.java:148)
at scala.collection.immutable.$colon$colon.<init>(List.scala:646)
at scala.collection.immutable.List.$colon$colon(List.scala:97)
at fs2.data.cbor.low.internal.ItemValidator$.validateChunk$1(ItemValidator.scala:138)
at fs2.data.cbor.low.internal.ItemValidator$.validate$1(ItemValidator.scala:100)
at fs2.data.cbor.low.internal.ItemValidator$.validateChunk$1(ItemValidator.scala:138)
at fs2.data.cbor.low.internal.ItemValidator$.validate$1(ItemValidator.scala:100)
at fs2.data.cbor.low.internal.ItemValidator$.validateChunk$1(ItemValidator.scala:138)
at fs2.data.cbor.low.internal.ItemValidator$.validate$1(ItemValidator.scala:100)
at fs2.data.cbor.low.internal.ItemValidator$.validateChunk$1(ItemValidator.scala:138)
…
The first stack frames can be ignored, they're more or less random. Relevant is the back-and-forth between validate and validateChunk in ItemValidator. This occurs for at least arrays of known and unknown size, probably also maps. Test case used to obtain the stack trace (with -Xss1m):
test("should not overflow stack for large arrays") {
val length = 100000
val items = Stream.constant(CborItem.TextString("first"), length)
// (Stream(CborItem.StartIndefiniteArray) ++ items ++ Stream(CborItem.Break))
(Stream(CborItem.StartArray(length)) ++ items)
.through(validate[IO])
.compile
.drain
.attempt
.map(res => expect(res.isRight))
}
Just slapping a Pull.suspend(…) around each call to validate that occurs from within validateChunk fixes the test case. But I'm not (yet) sure whether that's a good solution with regards to both performance and whether other inputs would still cause problems.
When validating large CBOR arrays,
ItemValidatoris prone to stack overflows like this one:The first stack frames can be ignored, they're more or less random. Relevant is the back-and-forth between
validateandvalidateChunkinItemValidator. This occurs for at least arrays of known and unknown size, probably also maps. Test case used to obtain the stack trace (with-Xss1m):Just slapping a
Pull.suspend(…)around each call tovalidatethat occurs from withinvalidateChunkfixes the test case. But I'm not (yet) sure whether that's a good solution with regards to both performance and whether other inputs would still cause problems.