Accessing source context on errors

[CLOVIS-AI]

Hi,

I am working on a project with a complex parser that pulls data from Source. If incorrect data is fed in, the parser throws an exception. The problem is that it's very hard to know which part of the data was incorrect, since the exception contains no context.

The idea behind this feature would be to add access some kind of data so the parser can add the "source location" in the thrown exception. For example, the name of the file, and the line number and column number of the last byte read through the source. Maybe the contents of the source as well.

This feature seems to be needed by most tools that would use kotlinx-io, so I assume there is already a way to access this context, but I did not find it myself.

[fzhinkin]

Source itself does not track any position, so it always fail to read from the "head" of data. While underlying sources can be aware of their "cursor's" position (if it makes any sense for a particular source), it would require extension of the RawSource interface. It's not impossible, and we can introduce a default method that returns some context, but then we still have to map last read "position" from that context to a position with Source's buffer (which could be modified directly, without using Source's functions).

But I agree that providing better insights would be really nice.

[JakeWharton]

I would also recommend try/catching the raw parser, creating a new exception with surrounding context (such as the originating file name), and adding the parser exception as the cause.

I don't really see how line and column number can be done generally. It assumes the data is entirely text-based rather than byte-based or mixed media. There is no line and column when a PNG file fails to parse, for example.

For line/column numbers I would consider a wrapper RawSource that tracks newlines and adds that context when IOExceptions are thrown only for cases that are text-based. Plus it means we don't pay the cost of that tracking for sources which don't need it.

[CLOVIS-AI]

Thanks @JakeWharton. If I understand correctly, you mean something conceptually similar to java.io.LineNumberReader? If so, could this feature be added to the library?

[JakeWharton]

Yep! Didn't know that existed, but that's the concept for sure.

[fzhinkin]

That looks interesting, thanks for pointing to it! @CLOVIS-AI could you please also point us to a project where you'd like to use such a functionality, so we had a better understanding of possible use-cases?

[CLOVIS-AI]

I'm thinking of multiple such projects but the only one I can mention is this: https://gitlab.com/opensavvy/cel-kotlin

This is (or well, will be) a parser and interpreter for the CEL language. This particular project I mostly build to learn more about programming language design and implementation, so I'm purposefully implementing the parser directly using a Source, without using a grammar library.

Ideally, I'd like the parser to emit context about an error (file name, line number, column number, content of the current line) to make it easier for the user to understand what went wrong.

I expect a typical use-case for this feature would be reading configuration files or similar simple-but-custom formats.