Allow NaN attributes in datasets

[kwilcox]

Datasets with NaN value attributes currently raise a ValueError: Out of range float values are not JSON compliant error. This adds a starlette.responses.JsonResponse subclass that explicitly allows nan values. The class is taken straight from starlette and only changes the allow_nan parameter.

[jhamman]

Thanks @kwilcox - I'll let @abkfenris make the final call but this looks good to me.

[kwilcox]

Tests pass for me locally (admittedly on python 3.11), I'll poke around! Helps to be on the right branch... looking...

[abkfenris]

Tests pass for me locally (admittedly on python 3.11), I'll poke around! Helps to be on the right branch... looking...

And that's why I refreshed the PR before replying after a call went a lot longer than expected when I started to write a response.

It looks like it's hitting a different metadata test, which I'm guessing is also having a hard time with NaNs, and might require a deeper dive into the Zarr spec: https://github.com/xarray-contrib/xpublish/blob/ce9fa67b132cb78b795b9d1b18dbf7728d7f545a/tests/test_rest_api.py#L281-L284

@kwilcox I'd also love to hear how you're using xpublish at some point.

[kwilcox]

Well, the tests are passing. I didn't consult the Zarr spec on what to do with NaN values. I'm not a huge fan of the JSON response format this PR implements now... the JSON returned isn't valid JSON (it uses JavaScript NaN objects instead of null). I'll take a look at the Zarr docs and see what I can come up with for a better solution. I do have a bunch of datasets with NaN attribute values.

[abkfenris]

I was really hoping to quickly give high fives and approve it, but it looks like you found a really messy messy spot for a PR!

It sounds like there is a difference between what Zarr-Python does (mainly due to standard lib JSON), and the Zarr spec. It sounds like in the Zarr spec they are not currently happy with the handling of NaNs in attributes. Some Zarr-Python encoded datasets aren't openable in languages that have more restrictive JSON parsers,

A few entry points to the attribute encoding discussion that I found:

• Support for inf, nan, binary data in attributes zarr-developers/zarr-specs#141
• Special treatment of NaN in JSON user attributes zarr-developers/zarr-python#412

[mwengren]

@kwilcox I'd also love to hear how you're using xpublish at some point

Me too!

[abkfenris]

Since this matches up with how ds.to_zarr() formats .zmetadata, and even if it's not the most compatible, it's a known incompatibility with v2, so that works for me.

@jhamman it might be worth pulling Zarr support out into it's own plugin. Then we can tinker with compatibility options and follow v3 changes at a different pace.

[abkfenris]

Thanks Kyle!