Is this great project still maintained?

[derolf]

I just did a quick benchmark of zipson vs various serialisers + compressors.

The results are (length in bytes, time in ms, 100 iterations to warmup and 100 iterations to measure):

    ┌─────────┬───────────────────┬────────┬─────────────────────┬──────────────────────┐
    │ (index) │       name        │ length │     encodeTime      │      decodeTime      │
    ├─────────┼───────────────────┼────────┼─────────────────────┼──────────────────────┤
    │    0    │  'zipson brotli'  │  417   │     2.52860035      │      0.12007231      │
    │    1    │ 'zipson deflate'  │  475   │ 0.20293473999999997 │      0.11486058      │
    │    2    │ 'msgpack brotli'  │  576   │     3.92180084      │      0.05375433      │
    │    3    │   'json brotli'   │  599   │  4.567029679999999  │      0.07787885      │
    │    4    │ 'msgpack deflate' │  627   │      0.2011609      │      0.06081639      │
    │    5    │   'zipson lz4'    │  642   │     0.38702751      │  0.6388705200000001  │
    │    6    │   'msgpack lz4'   │  690   │ 0.6580354100000001  │      0.6165729       │
    │    7    │  'json deflate'   │  711   │ 0.10077317999999999 │ 0.053146879999999994 │
    │    8    │     'zipson'      │  1121  │     0.14132796      │      0.09936688      │
    │    9    │    'json lz4'     │  1123  │     0.13947896      │      0.47471737      │
    │   10    │     'msgpack'     │  1640  │     0.09119614      │      0.03184669      │
    │   11    │      'json'       │  2328  │     0.02867283      │      0.02063535      │
    └─────────┴───────────────────┴────────┴─────────────────────┴──────────────────────┘

So, zipson combined with deflate is fast and saves a lot of storage!

For the sake of our climate, you should continue your great work!!!

[derolf]

Ah, just figured out that you reduce precision of floating point numbers. When turning that off, zipson drops down to:

    ┌─────────┬───────────────────┬────────┬──────────────────────┬──────────────────────┐
    │ (index) │       name        │ length │      encodeTime      │      decodeTime      │
    ├─────────┼───────────────────┼────────┼──────────────────────┼──────────────────────┤
    │    0    │ 'msgpack brotli'  │  543   │  3.9494768199999997  │      0.0487283       │
    │    1    │   'json brotli'   │  596   │      3.78558768      │      0.05463344      │
    │    2    │ 'msgpack deflate' │  625   │ 0.15624016000000002  │      0.05078611      │
    │    3    │  'zipson brotli'  │  645   │      3.1058089       │      0.13496444      │
    │    4    │   'msgpack lz4'   │  690   │      0.20149049      │      0.49015607      │
    │    5    │  'json deflate'   │  704   │      0.08844158      │       0.046485       │
    │    6    │ 'zipson deflate'  │  761   │      0.4224007       │      0.23291225      │
    │    7    │    'json lz4'     │  1106  │      0.13270977      │      0.46283148      │
    │    8    │   'zipson lz4'    │  1141  │      0.41867841      │      0.70976352      │
    │    9    │     'zipson'      │  1639  │ 0.18446820000000003  │      0.10465664      │
    │   10    │     'msgpack'     │  1640  │      0.08509001      │ 0.039339019999999995 │
    │   11    │      'json'       │  2335  │ 0.025989619999999998 │      0.02246419      │
    └─────────┴───────────────────┴────────┴──────────────────────┴──────────────────────┘

[jgranstrom]

@derolf thank you for doing the benchmarks! Looking back into this a bit after not being able to put much, if any, time into it for a while.

If you would somehow have the benchmark code you ran available still I would love to check it out, when I reviewed it again Zipson+GeneralPurposeCompression pretty much always outperforms the general purpose alone even when not having any floating points in the data, for “commonly json structured” data.

I had some ideas to implement for an update so any input you still have around would be helpful for sure, thank you!