Skip to content

palettize fails in numpy2 #175

@gerritholl

Description

@gerritholl

Describe the bug

trollflow2.colormap.palettize fails in numpy2 if the bins span the entire range of the corresponding dtype.

To Reproduce

import numpy as np
import trollimage.colormap

arr = np.linspace(0, 255, 100, dtype=np.uint8).reshape(10, 10)
colors = np.linspace(0, 255, 768).reshape(256, 3)
values = np.arange(0, 256, dtype=np.uint8)

trollimage.colormap.palettize(arr, colors, values)

Expected behavior

I expect this to complete without exception, like it does when using numpy 1.24.4.

Actual results

It fails with ValueError: bins must be monotonically increasing or decreasing.

/data/gholl/checkouts/trollimage/trollimage/colormap.py:182: RuntimeWarning: overflow encountered in scalar add
  outside_range_bin = max(np.nanmax(arr), values.max()) + 1
Traceback (most recent call last):
  File "/data/gholl/checkouts/protocode/mwe/trollimage-np2-palettize.py", line 8, in <module>
    trollimage.colormap.palettize(arr, colors, values)
  File "/data/gholl/checkouts/trollimage/trollimage/colormap.py", line 164, in palettize
    return _palettize(arr, values), tuple(colors)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/gholl/checkouts/trollimage/trollimage/colormap.py", line 174, in _palettize
    new_arr = _digitize_array(arr, values)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/gholl/checkouts/trollimage/trollimage/colormap.py", line 190, in _digitize_array
    new_arr = np.digitize(arr.ravel(), bins, right=right)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/gholl/mambaforge/envs/py312/lib/python3.12/site-packages/numpy/lib/_function_base_impl.py", line 5867, in digitize
    raise ValueError("bins must be monotonically increasing or decreasing")
ValueError: bins must be monotonically increasing or decreasing

Environment Info:

  • trollimage version: v1.24.0-7-gc665c4d

Additional context

The ValueError is due to a numpy2-incompatibility in trollimage:

https://github.com/pytroll/trollimage/blob/main/trollimage/colormap.py#L182

outside_range_bin = max(np.nanmax(arr), values.max()) + 1

numpy 1.24: uint8(255) + 1 == int64(256)

numpy 2.0: uint8(255) + 1 = uint8(0)

This bug is triggered in Satpy when loading the NWCSAF GEO cloudtype composite (see pytroll/satpy#2874 (comment)).

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions