Strings that are valid numbers in scientific notation get transformed when converting XML into JSON

[oleg-gr]

If you have a string in your XML, that also happens to be a valid number in scientific notation, it will be transformed into a java representation of that number when XML is transformed into JSONObject:

XML.toJSONObject("<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n" +
        "<str1>045e5000</str1>\n" +
        "<str2>45e5000</str2>\n" +
        "<str3>045e50</str3>\n" +
        "<str4>45e5</str4>");

will return

{"str1":"045e5000","str4":4500000,"str3":4.5E51,"str2":"45e5000"}

I understand that vanilla XML doesn't have a way to define types, but I still think this is a very unpredictable behavior. I am not sure how to solve it. Possibly, do not convert e notations and leave them as strings?

[stleary]

@oleg-gr, Agreed, but I think you would get the same result with this code:

        JSONObject jsonObject  = new JSONObject("\"str1\":045e5000,\"str2\":45e5000,\"str3\":045e50,\"str4\":45e5}");

I think the numbers with exponent 5000 could not be converted to Double, so were handled as String. This seems reasonable to me.

[oleg-gr]

@stleary, thank you for the reply.

Yeah, I know that the numbers with huge exponent weren't converted because they were too large, but in fact, in my opinion, it's a more expected behaviour. I am more concerned with other values being converted to Double instead of String since the original value gets mangled completely in some cases ("45e5" turns into 4500000 instead of staying "45e5"). Since as I was saying, in my case, those were intended to be strings (e.g. they are IDs or something else that just happens to have 'e' in them), I was surprised at this behaviour, but I see where it's coming from.

Since xml doesn't differentiate types, how do we deal with it? Why is it assumed that those should be tried to be converted as numbers getting the result like this:

new org.json.JSONObject("{\"str1\":045e5000,\"str2\":45e5000,\"str3\":045e50,\"str4\":45e5}");

and not the result like this:

new org.json.JSONObject("{\"str1\":\"045e5000\",\"str2\":\"45e5000\",\"str3\":\"045e50\",\"str4\":\"45e5\"}");

?

Do you think this is expected behaviour? Or do you think that if a number contains a letter 'e', it's more likely it's a string, rather than a number in scientific notation? I am not as familiar with xml and json format specifications, so your input on this would be appreciated.

[stleary]

Point taken. The rules are pretty straightforward, but the documentation should be clear. I will take an action item to update the JavaDocs and/or the Wiki accordingly. Til then, please see the diagrams on http://json.org, and recall from https://www.ietf.org/rfc/rfc4627.txt, "An implementation may set limits on the range of numbers."

Internally, our rule is:

• If the string seems like it might be a number, try to convert it using built-in methods.
• If successful, it is a number.
• Otherwise, it is a String.

We use Double.valueOf(String), new Long(String), and Integer.valueOf(int) to attempt the conversions. Officially, the lib only supports these 3 numeric types. Recently we added some support for BigNumber and BigDecimal, but they are not foundational, and are not considered when parsing JSON documents.

[oleg-gr]

Ok sounds good. Any idea of a workaround for cases like mine, when those are actually intended to stay as strings? Unfortunately, I don't have access to the XML I am trying to convert.

One more opinion on the topic. I feel that it's not fair to change the representation of the number. If XML says '4e5', I feel like it's not fair to write it out as 40000 since '4e5'.toString() and 40000.toString() produce different results (hope it makes sense). Maybe, when you try Double.valueOf(X), new Long(X), and Integer.valueOf(X), you should also compare the toString() output of the resulting value with X.toString(), and if they are different, treat X as a string?

[stleary]

The most straightforward option is to fork the project and change to your preferred behavior. In XML.parse() there are 2 places where JSONObject.stringToValue() is called. Replace them with your own method.

Regarding the representation, you can submit a pull request and make a strong case for why and how it should be changed. However, changing behavior of existing code that is not clearly broken has a very high bar for acceptance.