Some variables from some files can't be loaded due to buffer too small

[valeriupredoi]

I have added an MRE that runs as a test in this branch (that we can keep later as a regular test that passes, once we fix the issue). Here as a normal script:

import os

import pyfive
import s3fs


def _load_nc_file(ncvar):
    """
    Get the netcdf file and its b-tree.

    Fixture to test loading an issue file.
    """
    issue_file = "da193a_25_6hr_t_pt_cordex__198807-198807.nc"
    storage_options = {
        'key': "f2d55c6dcfc7618b2c34e00b58df3cef",
        'secret': "$/'#M{0{/4rVhp%n^(XeX$q@y#&(NM3W1->~N.Q6VP.5[@bLpi='nt]AfH)>78pT",
        'client_kwargs': {'endpoint_url': "https://uor-aces-o.s3-ext.jc.rl.ac.uk"},  # final proxy
    }
    test_file_uri = os.path.join(
        "bnl",
        issue_file
    )
    fs = s3fs.S3FileSystem(**storage_options)
    s3file = fs.open(test_file_uri, 'rb')
    nc = pyfive.File(s3file)
    ds = nc[ncvar]

    return ds

it returns:

Traceback (most recent call last):
  File "/home/valeriu/pyfive/tests/test_buffer_issue.py", line 31, in <module>
    _load_nc_file('m01s30i111')
    ~~~~~~~~~~~~~^^^^^^^^^^^^^^
  File "/home/valeriu/pyfive/tests/test_buffer_issue.py", line 26, in _load_nc_file
    ds = nc[ncvar]
         ~~^^^^^^^
  File "/home/valeriu/miniconda3/envs/pyactive4/lib/python3.13/site-packages/pyfive/high_level.py", line 94, in __getitem__
    return Dataset(obj_name, DatasetID(dataobjs), self)
                             ~~~~~~~~~^^^^^^^^^^
  File "/home/valeriu/miniconda3/envs/pyactive4/lib/python3.13/site-packages/pyfive/h5d.py", line 93, in __init__
    self._meta = DatasetMeta(dataobject)
                 ~~~~~~~~~~~^^^^^^^^^^^^
  File "/home/valeriu/miniconda3/envs/pyactive4/lib/python3.13/site-packages/pyfive/h5d.py", line 593, in __init__
    self.attributes = dataobject.get_attributes()
                      ~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/valeriu/miniconda3/envs/pyactive4/lib/python3.13/site-packages/pyfive/dataobjects.py", line 154, in get_attributes
    more_attrs = self._get_attributes_from_attr_info(attrs, attr_info)
  File "/home/valeriu/miniconda3/envs/pyactive4/lib/python3.13/site-packages/pyfive/dataobjects.py", line 180, in _get_attributes_from_attr_info
    name, value = self._parse_attribute_msg(data,0)
                  ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
  File "/home/valeriu/miniconda3/envs/pyactive4/lib/python3.13/site-packages/pyfive/dataobjects.py", line 237, in _parse_attribute_msg
    value = self._attr_value(dtype, buffer, items, offset)
  File "/home/valeriu/miniconda3/envs/pyactive4/lib/python3.13/site-packages/pyfive/dataobjects.py", line 273, in _attr_value
    value = np.frombuffer(buf, dtype=dtype, count=count, offset=offset)
ValueError: buffer is smaller than requested size

pyfive=0.6.0 and 0.5.1 from conda-forge or PyPI; also, I cleaned up my swap.

GHA test output https://github.com/NCAS-CMS/pyfive/actions/runs/17982539313/job/51152232062?pr=99

Closes #100

[bnlawrence]

Is this a test for an issue that hasn't yet been raised?

[valeriupredoi]

I was planning on creating the issue after I've tested with the file local to FS on POSIX - that way we know it's not something to do with s3fs. But my day ended before I could do that, will do tomorrow 👍🍺

[valeriupredoi]

I was also keen on reproducing the issue in a different environment other than my machine, if that had passed on GHA machines this would have been a very short lived PR 😁

[valeriupredoi]

fix by @kmuehlbauer in #102 and I have also ported the test here in that PR so this can now be closed