Skip to content

Using a forward slash in a node names of a DataTree causes a RecursionError #9978

Open
@dawiedotcom

Description

@dawiedotcom

What happened?

When creating an xarray.DataTree object, passing the optional children parameter with / in a key of the dictionary produces an infinite recursion loop.

It appears that a ValueError: node names cannot contain forward slashes is raised, but the program then gets stuck in a mutual recursive loop between

child._set_parent(new_parent=self, child_name=name) # xarray/core/treenode.py line 333

and

self._attach(new_parent, child_name) # xarray/core/treenode.py line 115

This happens before the ValueError can be handled by the call to xarray.DataTree.

What did you expect to happen?

Either of the following:

  • Just raise ValueError("node names cannot contain forward slashes"), without the recursion issue.
  • Even better: using / in the keys of the children parameter produces nested DataTree objects. This was be behaviour in xarray-contrib/datatree.

Minimal Complete Verifiable Example

from xarray import DataTree

# Wrapping this in try, except handles the RecursionError, not the ValueError
dt = DataTree(children={'a/b': DataTree()})

MVCE confirmation

  • Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
  • Complete example — the example is self-contained, including all data and the text of any traceback.
  • Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
  • New issue — a search of GitHub Issues suggests this is not a duplicate.
  • Recent environment — the issue occurs with the latest version of xarray and its dependencies.

Relevant log output

Traceback (most recent call last):
  File ".venv/lib/python3.13/site-packages/xarray/core/treenode.py", line 182, in children
    child._set_parent(new_parent=self, child_name=name)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.13/site-packages/xarray/core/treenode.py", line 115, in _set_parent
    self._attach(new_parent, child_name)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.13/site-packages/xarray/core/treenode.py", line 159, in _attach
    self._post_attach(parent, child_name)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.13/site-packages/xarray/core/treenode.py", line 723, in _post_attach
    _validate_name(name)  # is this check redundant?
    ~~~~~~~~~~~~~~^^^^^^
  File ".venv/lib/python3.13/site-packages/xarray/core/treenode.py", line 676, in _validate_name
    raise ValueError("node names cannot contain forward slashes")
ValueError: node names cannot contain forward slashes

During handling of the above exception, another exception occurred:

... many repetitions of 'ValueError: node names cannot contain forward slashes' and it's stack trace

Traceback (most recent call last):
  File ".venv/lib/python3.13/site-packages/xarray/core/treenode.py", line 182, in children
    child._set_parent(new_parent=self, child_name=name)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.13/site-packages/xarray/core/treenode.py", line 115, in _set_parent
    self._attach(new_parent, child_name)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.13/site-packages/xarray/core/treenode.py", line 152, in _attach
    self._pre_attach(parent, child_name)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.13/site-packages/xarray/core/datatree.py", line 528, in _pre_attach
    check_alignment(path, node_ds, parent_ds, self.children)
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.13/site-packages/xarray/core/datatree.py", line 149, in check_alignment
    align(node_ds, parent_ds, join="exact", copy=False)
    ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.13/site-packages/xarray/core/alignment.py", line 883, in align
    aligner.align()
    ~~~~~~~~~~~~~^^
  File ".venv/lib/python3.13/site-packages/xarray/core/alignment.py", line 572, in align
    self.find_matching_indexes()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File ".venv/lib/python3.13/site-packages/xarray/core/alignment.py", line 254, in find_matching_indexes
    obj_indexes, obj_index_vars = self._normalize_indexes(obj.xindexes)
                                  ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
  File ".venv/lib/python3.13/site-packages/xarray/core/alignment.py", line 215, in _normalize_indexes
    for idx, index_vars in Indexes(xr_indexes, xr_variables).group_by_index():
                           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File ".venv/lib/python3.13/site-packages/xarray/core/indexes.py", line 1624, in group_by_index
    for i, index in self._id_index.items():
                    ^^^^^^^^^^^^^^
  File ".venv/lib/python3.13/site-packages/xarray/core/indexes.py", line 1515, in _id_index
    self.__id_index = {id(idx): idx for idx in self.get_unique()}
                                               ~~~~~~~~~~~~~~~^^
RecursionError: maximum recursion depth exceeded

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "xarray_datatree_recursion_loop.py", line 2, in <module>
    DataTree(children={'a/b': DataTree(name='leaf')})
    ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.13/site-packages/xarray/core/datatree.py", line 506, in __init__
    super().__init__(name=name, children=children)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.13/site-packages/xarray/core/treenode.py", line 691, in __init__
    super().__init__(children=children)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.13/site-packages/xarray/core/treenode.py", line 87, in __init__
    self.children = {name: child.copy() for name, child in children.items()}
    ^^^^^^^^^^^^^
  File ".venv/lib/python3.13/site-packages/xarray/core/common.py", line 333, in __setattr__
    object.__setattr__(self, name, value)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.13/site-packages/xarray/core/treenode.py", line 187, in children
    self.children = old_children
    ^^^^^^^^^^^^^
  File ".venv/lib/python3.13/site-packages/xarray/core/common.py", line 333, in __setattr__
    object.__setattr__(self, name, value)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.13/site-packages/xarray/core/treenode.py", line 187, in children
    self.children = old_children
    ^^^^^^^^^^^^^

... many repetitions jumping between the previous two calls

  File ".venv/lib/python3.13/site-packages/xarray/core/common.py", line 333, in __setattr__
    object.__setattr__(self, name, value)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.13/site-packages/xarray/core/treenode.py", line 187, in children
    self.children = old_children
    ^^^^^^^^^^^^^

  File ".venv/lib/python3.13/site-packages/xarray/core/treenode.py", line 115, in _set_parent
    self._attach(new_parent, child_name)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.13/site-packages/xarray/core/treenode.py", line 152, in _attach
    self._pre_attach(parent, child_name)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.13/site-packages/xarray/core/datatree.py", line 521, in _pre_attach
    if name in parent.dataset.variables:
               ^^^^^^^^^^^^^^
  File ".venv/lib/python3.13/site-packages/xarray/core/datatree.py", line 614, in dataset
    return self._to_dataset_view(rebuild_dims=True, inherit=True)
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.13/site-packages/xarray/core/datatree.py", line 553, in _to_dataset_view
    coord_vars = self._coord_variables if inherit else self._node_coord_variables
                 ^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.13/site-packages/xarray/core/datatree.py", line 541, in _coord_variables
    *(p._node_coord_variables_with_index for p in self.parents),
                                                  ^^^^^^^^^^^^
  File ".venv/lib/python3.13/site-packages/xarray/core/treenode.py", line 338, in parents
    return tuple(self._iter_parents())
  File ".venv/lib/python3.13/site-packages/xarray/core/treenode.py", line 305, in _iter_parents
    node: Tree | None = self.parent
                        ^^^^^^^^^^^
RecursionError: maximum recursion depth exceeded

Anything else we need to know?

I tested this on the latest main branch in a new conda environment, but it appears the issue is present in every version of xarray since the introduction of DataTree.

Environment

INSTALLED VERSIONS

commit: None
python: 3.13.1 (main, Dec 4 2024, 18:05:56) [GCC 14.2.1 20240910]
python-bits: 64
OS: Linux
OS-release: 6.12.9-arch1-1
machine: x86_64
processor:
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: ('en_US', 'UTF-8')
libhdf5: 1.14.2
libnetcdf: 4.9.4-development

xarray: 2025.1.1
pandas: 2.2.3
numpy: 2.2.1
scipy: 1.15.1
netCDF4: 1.7.2
pydap: None
h5netcdf: None
h5py: None
zarr: None
cftime: 1.6.4.post1
nc_time_axis: None
iris: None
bottleneck: None
dask: None
distributed: None
matplotlib: 3.10.0
cartopy: None
seaborn: None
numbagg: None
fsspec: None
cupy: None
pint: None
sparse: None
flox: None
numpy_groupies: None
setuptools: None
pip: None
conda: None
pytest: 8.3.4
mypy: None
IPython: None
sphinx: None

Metadata

Metadata

Assignees

Labels

bugtopic-DataTreeRelated to the implementation of a DataTree class

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions