Skip to content

Incremental parsing does not work when parse_some() is called multiple times. #576

@ergonjona

Description

@ergonjona

We use jsoncons to parse JSON in HTTP bodies and want to support streaming as much as possible.
In jsoncons version 0.178.0 using the old incremental parsing methods we were able to parse JSON like so:

void parse_stream(std::vector<std::string_view> chunks, bool final) {
  for ( auto chunk : chunks ) {
    parser.update(chunk);
    parser.parse_some(decoder);
  }
  if ( final ) {
    parser.finish_parse(); 
    parser.check_done();
  }
  ...
}

This function would be called multiple times with some chunks and final == false and finally with zero or more chunks and final == true.

Since jsoncons 1.0.0 this is no longer possible.
I changed our code to use the new incremental parsing methods and it does still not work.
As long as the whole JSON string can be parsed in one call of parser.parse_some() everything works.
When we try to call parser.parse_some() multiple times an exception is thrown (some kind of syntax error). If we are not allowed to call parser.parse_some() multiple times we need to buffer all chunks before parsing, which destroys the streaming capability of our code.

I wrote a test program that reproduces the bug: https://gist.github.com/ergonjona/514e3b2581d96a78b765d153b4c5e556

  • Compiler: clang++ 18.1.8
  • Architecture: x64
  • Operating system: Arch Linux
  • jsoncons version 1.0.0

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions