Skip to content

Appending to re-opened zarr array does not respect write_empty_chunks #2931

Open
@ianhi

Description

@ianhi

Zarr version

3.0.7.dev3+g9e8b50ae

Numcodecs version

0.15.1

Python Version

3.13.2

Operating System

mac

Installation

from source

Description

When you open an existing array in order to resize and write new values to it the write_empty_chunks config is no longer respected.

Steps to reproduce

import os
import zarr

z = zarr.create_array(
    store="appending.zarr",
    name="test",
    shape=(2,),
    dtype=int,
    chunks=(1,),
    config={"write_empty_chunks": True},
    overwrite=True,
    fill_value=0,
    zarr_format=3,
)

z[:] = 0

# if you comment out this re-opening then everything works as intended
z = zarr.open(
    "appending.zarr", path="test", config={"write_empty_chunks": True}, fill_value=0
)
print(os.listdir("appending.zarr/test/c/"))

z.resize((4,))

print(os.listdir("appending.zarr/test/c/"))
l = os.listdir("appending.zarr/test/c/")
print(l)
assert l == ['0', '1',]

# now put a value into the newly appended chunks
z[2:] = 0
l = os.listdir("appending.zarr/test/c/")
print(l)
# assert l == ['0', '1', '2', '3']


# putting in a value of zero makes all empty chunks be removed from disk
z[:] = 0

l = os.listdir("appending.zarr/test/c/")
print(l)
# assert l == ['0', '1', '2', '3']

gives

['0', '1']
['0', '1']
['0', '1']
['0', '1']
[]

not only are chunks 2 and 3 not written, but unexpectedly chunks 0 and 1 are removed from disk.

if you comment out the zarr.open... line then the output looks like:

['0', '1']
['0', '1']
['0', '1']
['0', '1', '3', '2']
['0', '1', '3', '2']

Additional output

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugPotential issues with the zarr-python library

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions