H5Fis_accessible(): unable to determine if file is accessible as HDF5

[rkouznetsov]

Hi,
I run some nested command with cdo on a computer with lustre file system. cdo is vanilla opensuse (the issue isalso reproduicble with Debian and Ubuntu builds) wrapped into a singularity contaner.

$ cdo --version
Climate Data Operators version 2.6.3 (https://mpimet.mpg.de/cdo)
System: x86_64-suse-linux-gnu
CXX Compiler: g++ -std=gnu++20 -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -flto=auto -g -O3 -fno-strict-aliasing -I/usr/include/magics -fopenmp -pthread
CXX version : g++ (SUSE Linux) 13.4.0
CXX library :
C Compiler: gcc -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -flto=auto -g -O3 -fno-strict-aliasing -I/usr/include/magics -fopenmp -pthread -pthread
C version : gcc (SUSE Linux) 13.4.0
F77 Compiler:  -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -flto=auto -g 
Features: 55GB 24threads c++20 OpenMP45 pthreads HDF5 NC4/HDF5/threadsafe dap sz udunits2 proj xml2 magics curl sse4_2
Libraries: yac/3.13.1 NetCDF/4.10.0 HDF5/1.14.6 proj/9.5.1 xml2/2.13.8 curl/8.14.1 magics/4.16.0
CDI data types: SizeType=size_t
CDI file types: srv ext ieg grb1 grb2 nc1 nc2 nc4 nc4c nc5 nczarr 
     CDI library version : 2.6.3
 cgribex library version : 2.3.1
 ecCodes library version : 2.47.0
  NetCDF library version : 4.10.0 of Apr  8 2026 01:48:32 $
    exse library version : 2.1.0
    FILE library version : 1.9.2

The output created, but I get

cdo    expr: Processed 107310000 values from 1 variable over 365 timesteps [11.24s 1845MB]
cdo(1) remap: Process started
cdo(2) selname: Process started
cdo(3) mergetime: Process started
HDF5-DIAG: Error detected in HDF5 (1.14.6) thread 1:
  #000: /home/abuild/rpmbuild/BUILD/hdf5-1.14.6-build/hdf5-1.14.6/src/H5F.c line 496 in H5Fis_accessible(): unable to determine if file is accessible as HDF5
    major: File accessibility
    minor: Not an HDF5 file
  #001: /home/abuild/rpmbuild/BUILD/hdf5-1.14.6-build/hdf5-1.14.6/src/H5VLcallback.c line 3913 in H5VL_file_specific(): file specific failed
    major: Virtual Object Layer
    minor: Can't operate on object
  #002: /home/abuild/rpmbuild/BUILD/hdf5-1.14.6-build/hdf5-1.14.6/src/H5VLcallback.c line 3848 in H5VL__file_specific(): file specific failed
    major: Virtual Object Layer
    minor: Can't operate on object
  #003: /home/abuild/rpmbuild/BUILD/hdf5-1.14.6-build/hdf5-1.14.6/src/H5VLnative_file.c line 344 in H5VL__native_file_specific(): error in HDF5 file check
    major: File accessibility
    minor: Can't get value
  #004: /home/abuild/rpmbuild/BUILD/hdf5-1.14.6-build/hdf5-1.14.6/src/H5Fint.c line 1055 in H5F__is_hdf5(): unable to open file
    major: File accessibility
    minor: Unable to initialize object
  #005: /home/abuild/rpmbuild/BUILD/hdf5-1.14.6-build/hdf5-1.14.6/src/H5FD.c line 787 in H5FD_open(): can't open file
    major: Virtual File Layer
    minor: Unable to open file
  #006: /home/abuild/rpmbuild/BUILD/hdf5-1.14.6-build/hdf5-1.14.6/src/H5FDsec2.c line 323 in H5FD__sec2_open(): unable to open file: name = 'v61-CAMAERAref3_dly7/aerocom3_SILAM_concdust25_2D_2019_daily.nc', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0
    major: File accessibility
    minor: Unable to open file
cdo    expr: Processed 107310000 values from 1 variable over 365 timesteps [10.85s 1845MB]

It looks like some race condition when one process/thread opens the file for writing, while other tries to check the file state on the disk...

I wonder if that message and/or the check makes any sense. If the file can be written properly, then there is no point to bug the user. Otherwise the program should crash properly...

Thank you!

[rkouznetsov]

UPD: The same container works just fine on the same data when working on a fast local filesystem.