Avoid duplicate Zarr array read

[dcherian]

We already get the underlying Zarr array in

xarray/xarray/backends/zarr.py

Lines 529 to 531 in bb8511e

	return FrozenDict(
	(k, self.open_store_variable(k, v)) for k, v in self.zarr_group.arrays()
	)

and then pass it to open_store_variable. Just pass that array down to ZarrArrayWrapper instead of reading it from the datastore again.

[rabernat]

This is the logical next step after #8297.

[jhamman]

Suggested change

	def __init__(self, variable_name, zarr_array, datastore):
	def __init__(self, zarr_array):

Fairly sure this should work.

[Illviljan]

Test this properly. I've not had good results doing this trick on a different datastore.

[dcherian]

thanks for the heads up! what issues did you see?

[Illviljan]

I'm not completely sure how the zarr backend is implemented, but in my case after xr.open_dataset() the datastore is closed and not needed anymore( until .compute()).

But in this case you're sending a reference from that store to the backendarray which will be lost once the store is closed.

I've gone the opposite way, in get_variables only the required keys are sent. Then only once the backendarray __getitem__ is triggered the arrays are loaded.

I noticed this quite quickly once I played around with the dataset and loaded some variables and if you haven't noticed it yet maybe the zarr backend does some interesting tricks to avoid the closed references.

[dcherian]

Yerah not sure. We do have distributed tests for reading zarr so presumably it's all OK

[jhamman]

Whoop whoop!