JSONFlattenerMaker: Speed up charsetFix.

[gianm]

JSON parsing has this function charsetFix that fixes up strings so they can round-trip through UTF-8 encoding without loss of fidelity. It was originally introduced to fix a bug where strings could be sorted, encoded, then decoded, and the resulting decoded strings could end up no longer in sorted order (due to character swaps during the encode operation).

The code has been in place for some time, and only applies to JSON. I am not sure if it needs to apply to other formats; it's certainly more difficult to get broken strings from other formats. It's easy in JSON because you can write a JSON string like "foo\uD900".

At any rate, this patch does not revisit whether charsetFix should be applied to all formats. It merely optimizes it for the JSON case. The function works by using CharsetEncoder.canEncode, which is a relatively slow method (just as expensive as actually encoding). This patch adds a short-circuit to skip canEncode if all chars in a string are in the basic multilingual plane (i.e. if no chars are surrogates).

Benchmarks:

master

Benchmark                              (discovery)  (readerTypeString)  Mode  Cnt     Score    Error  Units
JsonInputFormatBenchmark.parseAndRead        false              reader  avgt   10  2645.716 ± 24.261  ns/op

patch

Benchmark                              (discovery)  (readerTypeString)  Mode  Cnt     Score    Error  Units
JsonInputFormatBenchmark.parseAndRead        false              reader  avgt   10  2307.164 ± 36.656  ns/op

[cryptoe]

Thanks for the JMH benchmarks. I was worried for another O(N) iteration but I guess the overhead would be worthwhile since roundTrip is skipped.