Tracking: Stable 1.0

[zesterer]

A list of things that need doing before 1.0 can be released as stable. This list will grow as we become aware of more things or remember outstanding issues.

API

General

• Consider whether ParserSealed is necessary (perhaps doc(hidden) is sufficient for semver? Should we take a 'it's your fault if you don't read the docs' perspective?)
• The sync feature is broken. When enabled, it adds extra Send/Sync constraints to trait object parsers like recursive, thereby making it non-additive.
• Consider IterParser API overhaul from Add a way to turn a regular Parser into an IterParser so that chumsky can be used as a stream transformer #425
• Reconsider inclusion of the lexical crate (apparently it has soundness issues and is unmaintained?)
• Remove SimpleSpan::new in favour of Span::new, which is more general.

Combinators

• Merge map_*, try_map_*, foldl_* and foldr_* combinators into one (see Added map_with and try_map_with to replace existing mapper functions #537).
• Implement pratt parser combinator
  • Prefix unary
  • Postfix unary
  • Infix binary
  • Dynamic building of parsers (i.e: accept a Vec<Operator> or something of the sort, see here)
• Find a solution for leaky recursive (recursive-leak: fix leak by using middle-man #494 seems like a good candidate)
• Give the text parsers, such as text::ident a distinct type (instead of returning an impl Parser), allowing us to propagate trait bounds through properly

Inputs

• Consider implementation of CopyStream (that copies the iterator instead of caching tokens, more performant for iterators with very little state, like logos iterators)
• Consider naming of Stream: is IterInput more sensible?
• Should Input::spanned take a 'full input' span rather than an 'end of input' span? Might be more general.

Errors

• Unify 'expected' and label system (i.e: the former should be a special case of the latter)
• Ensure that all parsers necessarily provide at least one label (see select! when returning an error gives an empty expected #504 and First attempt at filter labels #538)
• Remove RichReason::Many

Extension API

• Initial implementation

Documentation

• Final pass to check correctness of documentation (some of it is from the 0.9 days)
• Nail down consistent terminology
  • 'token' and not 'character'/'element'
  • 'input' and not 'stream'/'iterator' (except when those terms are correct)
  • 'fatal' and 'recovered' errors, not 'primary' or 'secondary'

Examples

• Create a new example for pythonic indentation parsing with the new context-sensitive parsers

Guide

• Finish writing guide sections
• Final spellchecking/grammar pass
• Rewrite tutorial, including demonstration of:
  • Lexing
  • Spans
  • Error recovery

Other work

• Write migration guide (a good way to go about this might be porting a non-trivial parser and noting the major issues).
• Collect up all changes into the changelog (it's going to be big!)

[safasofuoglu]

Sad to see lexical being unsound and stale. parse_partial(&[u8]) really makes a performance difference. Can https://github.com/aldanor/fast-float-rust be considered as an alternative?

[zesterer]

I think the parse_partial function from fast-float would work nicely with chumsky, yes (the API is pretty much identical to the one exposed by regex-automata, which we use for the regex parser). As for integer parsing: I'm less certain, although there are less complexities around integer parsing so implementing our own fast integer parser might not be too difficult.

[sshine]

Do I understand correctly that the list of examples not yet ported to 1.0 are those in the examples/ sub-directory that do not have an [[example]] section? Update: My understanding is corrected; for example, foo is updated but does not have an [[example]] section.

[Zij-IT]

Sad to see lexical being unsound and stale. parse_partial(&[u8]) really makes a performance difference. Can https://github.com/aldanor/fast-float-rust be considered as an alternative?

lexical appears to be fixed and the maintainer is back (:

[zesterer]

That's good to know!