Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add configurable limit for the maximum number of bytes/chars of content to parse before failing #1046

Closed
cowtowncoder opened this issue Jun 12, 2023 · 3 comments
Labels
2.16 Issue planned (at earliest) for 2.16 processing-limits Issues related to limiting aspects of input/output that can be processed without exception

Comments

@cowtowncoder
Copy link
Member

cowtowncoder commented Jun 12, 2023

(note: part of #637)

Jackson 2.15 included a few processing limits that can be applied to limit processing for "too big content"; first focusing on general nesting depth and max. length of individual tokens.
While this is good first step, it also makes sense to offer a simple way to limit maximum content in total allowed to be read -- typically a maximum document size, but in case of line-delimited input, maximum streaming content.

The reasoning for addition of such feature is that although users can -- if they must -- implement this at yet lower level (length-limited InputStream, for example), there are some benefits from Jackson streaming component offering this:

  1. Less work for user (obviously), better accessibility leading to wider adoption and helping against possible DoS vectors
  2. Better integration via well-defined exception type common to constraints violations (StreamConstraintsException)
  3. More reliable limits when shared implementation used (i.e. less like users/devs implement faulty limits checks)

Note, too, that this feature significantly improves usefulness (or right now, lack thereof) of #863 to combine per-token limits with overall limits.

NOTE: the default setting for this limits should, however, be left as "unlimited": using anything else is likely to break some processing somewhere.
Limit has to be defined as 64-bit long (not int); default value to use then is likely Long.MAX_VALUE.

@cowtowncoder cowtowncoder added 2.16 Issue planned (at earliest) for 2.16 processing-limits Issues related to limiting aspects of input/output that can be processed without exception labels Jun 12, 2023
@pjfanning
Copy link
Member

pjfanning commented Jul 8, 2023

I guess built-in support would be good - but this limit is straightforward for users to apply themselves without waiting for a jackson solution.

If your input data is the form of

  • InputStream - try SizeLimitInputStream or equivalent
  • Reader - try SizeLimitReader or equivalent
  • String - just check length() of String
  • ByteBuffer - jackson-databind has ByteBufferBackedInputStream - you can write the ByteBuffer with this and then wrap the InputStream with the SizeLimitInputStream above
    • it may also be feasible to check the ByteBuffer capacity before reading from it

@cowtowncoder
Copy link
Member Author

Still hoping to get this in 2.16 as it would be the last processing limit formally requested. But we'll see :)

@cowtowncoder
Copy link
Member Author

Implemented for JSON backend (InputStream, Reader, non-blocking/async).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.16 Issue planned (at earliest) for 2.16 processing-limits Issues related to limiting aspects of input/output that can be processed without exception
Projects
None yet
Development

No branches or pull requests

2 participants