Exposing Zarr backend internals as semi-public API

[jhamman]

We recently built a prototype REST API for serving xarray datasets via a Fast-API application (see #3850 for more details). In the process of doing this, we needed to use a few internal functions in Xarray's Zarr backend:

from xarray.backends.zarr import (
    _DIMENSION_KEY,
    _encode_zarr_attr_value,
    _extract_zarr_variable_encoding,
    encode_zarr_variable,
)
from xarray.core.pycompat import dask_array_type
from xarray.util.print_versions import get_sys_info, netcdf_and_hdf5_versions

Obviously, none of these imports are really meant for use outside of Xarray's backends so I'd like to discuss how we may go about exposing these functions (or variables) as semi-public (advanced use) API features. Thoughts?

cc @rabernat

[rabernat]

Shouldn't this be solved by the entry points refactor of the backends? In the future, I imagine that the zarr backend for xarray will not live in xarray but rather in some third-party package. That package could expose all of these functions as public API.

[jhamman]

Yes, this is one (probable) path. I wanted to document that we were using these private API functions in an effort to highlight their usefulness and to document their off-piste use.

[shoyer]

The imports from xarray.backends.zarr all seem pretty reasonable to me. We could make those semi-public if you like. Of course, ideally the way to write netCDF style data would be codified in some third-party convention, not inside xarray.

I don't think it's a good idea to reuse the imports from xarray.core.pycompat or xarray.util.print_versions. I would just fork that code into your project rather than importing it.