Parsing of ServerSentEvents not working correctly when using reactive WebClient [SPR-16735]

[spring-projects-issues]

Harmen Renkema opened SPR-16735 and commented

We are running into some issues when streaming objects using WebClient from spring-webflux.

We've tracked them to the stream parsing in ServerSentEventHttpMessageReader.read(..).

The issues we're encountering:

1. for every event supplied by the backend we get 2 ServerSentEvents from WebClient. One populated with our data, and one fully populated with null values. (detailed example data below)
2. when the events supplied by the backend have newlines in their data section, every now and then the eventstream is split incorrectly. This results in incomplete json being fed into jackson, causing it to throw a JsonEOFException. This occurs at least once every couple of thousand of messages.

---

Example for the first issue:
We have a backend that provides the following stream.

id:1
event:myEvent
data:{
data:  "id" : 1,
data:  "type" : "myEvent",
data:  "body" : "{ \"drink\": \"coffee\", \"snack\": \"muffin\"}",
data:  "timestamp" : "2018-04-17T07:19:44.769Z"
data:}

id:2
event:myEvent
data:{
data:  "id" : 2,
data:  "type" : "myEvent",
data:  "body" : "{ \"drink\": \"coffee\", \"snack\": \"muffin\"}",
data:  "timestamp" : "2018-04-17T07:19:58.859Z"
data:}

id:3
event:myEvent
data:{
data:  "id" : 3,
data:  "type" : "myEvent",
data:  "body" : "{ \"drink\": \"coffee\", \"snack\": \"muffin\"}",
data:  "timestamp" : "2018-04-17T07:19:59.313Z"
data:}

When I use WebClient like this:

webClient.get()
	.accept(TEXT_EVENT_STREAM)
	.retrieve()
	.bodyToFlux(new ParameterizedTypeReference<ServerSentEvent<Object>>() {})
	.subscribe(System.out::println);

I end up with the following ServerSentEvents:

ServerSentEvent [id = '1', event='myEvent', retry=null, comment='null', data={id=1, type=myEvent, body={ "drink": "coffee", "snack": "muffin"}, timestamp=2018-04-17T07:19:44.769Z}]
ServerSentEvent [id = 'null', event='null', retry=null, comment='null', data=null]
ServerSentEvent [id = '2', event='myEvent', retry=null, comment='null', data={id=2, type=myEvent, body={ "drink": "coffee", "snack": "muffin"}, timestamp=2018-04-17T07:19:58.859Z}]
ServerSentEvent [id = 'null', event='null', retry=null, comment='null', data=null]
ServerSentEvent [id = '3', event='myEvent', retry=null, comment='null', data={id=3, type=myEvent, body={ "drink": "coffee", "snack": "muffin"}, timestamp=2018-04-17T07:19:59.313Z}]
ServerSentEvent [id = 'null', event='null', retry=null, comment='null', data=null]

---

Affects: 5.0.4

Issue Links:

• Decoding ServerSentEvent fails when using generic type [SPR-16744] #21285 Decoding ServerSentEvent fails when using generic type ("duplicates")

[spring-projects-issues]

Sebastiaan van Erk commented

If the input "data: foo\ndata: bar\n\n" is split into (for example 2) DataBuffers by the ReactiveHttpInputMessage as follows: "data:foo" and "\ndata: bar\n\n",
then the second data buffer will start with "\n". In combination with

.concatMap(ServerSentEventHttpMessageReader::splitOnNewline)

this will cause this starting "\n" to match:

.bufferUntil(line -> line.equals("\n"))

This will cause the code to incorrectly split the server sent event into two events.

[spring-projects-issues]

Rossen Stoyanchev commented

This appears to be a duplicate of #21285.