Significant increase in memory consumption StringDecoder when splitting input by delimiters.

[chali]

Affects: 6.1.1

---

Context

I have been moving from Spring 5 to Spring 6 and I noticed some of our tests were failing with OutOfMemoryError I have been able to tease out smaller code sample showing the problem.

val data = ClassPathResource("data.txt").inputStream.bufferedReader().readText()
val buffer: Publisher<DataBuffer> = Flux.just(
            DefaultDataBufferFactory.sharedInstance.wrap(data.toByteArray(
                Charset.forName("UTF-8"))))
StringDecoder.textPlainOnly().decode(buffer, ResolvableType.forClass(String::class.java), null, null).subscribe {
    println(it)
}

The bigger picture when this is happening in our case. We have a test for processing server sent events. Our test has canned events in a single file. Our tests were creating manually response from webclient with the whole file as response body. It seems like the individual event separation by delimiter into own DataBuffer is causing memory consumption. Previous version (5.3.31) of Spring we were using wasn't having this behavior

Reproducing example

1. clone https://github.com/chali/string-decode-memory-consumption/tree/master
2. run ./gradlew check

Memory flamegraphs

Spring 6.1.1 is showing following memory consuption

Spring 5.3.31

[sdeleuze]

@poutsma After a quick discussion with @simonbasle, it could be related to the fact we were using slice (so kind of views) in 5.3 and split in 6.1 with may involved more allocation. Could you please give it a look to see if there is something to refine in term of efficiency for that use case?

[poutsma]

@poutsma After a quick discussion with @simonbasle, it could be related to the fact we were using slice (so kind of views) in 5.3 and split in 6.1 with may involved more allocation.

The result of a split operation is as just as much a view as the result of a slice is, split is just more restrictive regarding the scope of the view.

It turned out to be a bug in reducing the capacity of the remainder buffer in DefaultDataBuffer::split. Preparing a fix now.

[chali]

Thank you for quick fix!