scala.util.parsing.combinator.JavaTokenParsers.floatingPointNumber Incorrect

[scabug]

The floating point parser in scala.util.parsing.combinator.JavaTokenParsers appears to be incorrect. The function in the code is:

def floatingPointNumber: Parser[String] = 
    """-?(\d+(\.\d*)?|\d*\.\d+)[eEfFdD]?([+-]?\d+)?""".r

However, this regular expression fails to correctly parse all valid floats. For example, 9e4f, which is valid in java, fails with the above regex. Even worse, it allows through invalid expressions. For example, 9+3, which is not a parsable float, is successfully captured by the above regex.

I believe the regular expression should be modified to allow trailing exponents only if "e" or "E" trails the first number sequence; to allow "D"/"d"/"F"/"f" if there is an exponent; and to allow an optional "+" at the front of the expression. I am not a regex expert, but I believe the following definition will satisfy the above requirements:

def floatingPointNumber: Parser[String] =
    """[+-]?(\d+(\.\d*)?|\d*\.\d+)([eE][+-]?\d+)?[fFdD]?""".r

[scabug]

Imported From: https://issues.scala-lang.org/browse/SI-1547?orig=1
Reporter: crf (rawls)

[scabug]

Mohsen Lesani [X] (lesani) said:

Please also supply a complete example.

[scabug]

@rkuhn said:
I hit the same problem a few weeks back but didn't get around to reporting it. The above solution is better than the current state, but still not complete wrt. the Java Language Specification. I would propose to add the following methods to JavaTokenParsers:

    def decimalIntegerLiteral: Parser[String] =
      """-?(0|[1-9]\d*)[lL]?""".r
    def octalIntegerLiteral: Parser[String] =
      """-?0[0-7]+[lL]?""".r
    def hexIntegerLiteral: Parser[String] =
      """-?0[xX][0-9a-fA-F]+[lL]?""".r
    def floatingPointLiteral: Parser[String] =
      ("-?(" +
      """(\d+\.\d*|\.\d+)([eE][+-]?\d+)?[fFdD]?|""" +
      """\d+[eE][+-]?\d+[fFdD]?|""" +
      """\d+([eE][+-]?\d+)?[fFdD]""" +
      ")").r

The integers can be parsed using java.lang.Integer.decode(), parseInt() does not handle the radix prefix. I've broken up the float expression for readability, but I hope that this does not incur a runtime cost.

[scabug]

@dcsobral said:
And now Java has added more number formats to the fold... Would you care to update your additions, Roland? Though I'm concerned that pre-JDK7 java will not be able to parse the new formats. To which side should we err?

[scabug]

@SethTisue said:
The parser combinators library is now community-maintained. Issues with it are now tracked at scala/scala-parser-combinators#61 instead of here in the Scala JIRA.

Interested community members: if you consider this issue significant, feel free to open a new issue for it on GitHub, with links in both directions.

[Philippus]

the initial bug reported looks like it was fixed in scala/scala@b0de8aa