Skip to content

CBOR's ItemValidator crashes with stack overflow #760

@ybasket

Description

@ybasket

When validating large CBOR arrays, ItemValidator is prone to stack overflows like this one:

java.lang.StackOverflowError
	at java.base/java.lang.invoke.Invokers.checkGenericType(Invokers.java:541)
	at scala.runtime.Statics.releaseFence(Statics.java:148)
	at scala.collection.immutable.$colon$colon.<init>(List.scala:646)
	at scala.collection.immutable.List.$colon$colon(List.scala:97)
	at fs2.data.cbor.low.internal.ItemValidator$.validateChunk$1(ItemValidator.scala:138)
	at fs2.data.cbor.low.internal.ItemValidator$.validate$1(ItemValidator.scala:100)
	at fs2.data.cbor.low.internal.ItemValidator$.validateChunk$1(ItemValidator.scala:138)
	at fs2.data.cbor.low.internal.ItemValidator$.validate$1(ItemValidator.scala:100)
	at fs2.data.cbor.low.internal.ItemValidator$.validateChunk$1(ItemValidator.scala:138)
	at fs2.data.cbor.low.internal.ItemValidator$.validate$1(ItemValidator.scala:100)
	at fs2.data.cbor.low.internal.ItemValidator$.validateChunk$1(ItemValidator.scala:138)
…

The first stack frames can be ignored, they're more or less random. Relevant is the back-and-forth between validate and validateChunk in ItemValidator. This occurs for at least arrays of known and unknown size, probably also maps. Test case used to obtain the stack trace (with -Xss1m):

  test("should not overflow stack for large arrays") {
    val length = 100000
    val items = Stream.constant(CborItem.TextString("first"), length)
    // (Stream(CborItem.StartIndefiniteArray) ++ items ++ Stream(CborItem.Break))
    (Stream(CborItem.StartArray(length)) ++ items)
      .through(validate[IO])
      .compile
      .drain
      .attempt
      .map(res => expect(res.isRight))
  }

Just slapping a Pull.suspend(…) around each call to validate that occurs from within validateChunk fixes the test case. But I'm not (yet) sure whether that's a good solution with regards to both performance and whether other inputs would still cause problems.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingcbor

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions