-
Notifications
You must be signed in to change notification settings - Fork 78
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for saving the index produced with a full GRIB scan at open. #20
Comments
I think it's worth mentioning that it probably should not fail if it cannot write the index file - the GRIB file could be in a read-only directory, in which case it would not be possible to write the index to the same place. |
The implementation is still wasteful and very fragile, but the index is saved to and read from disk. |
I just want to bump @iainrussell point above-- at present, it does not fail when reading from a read-only directory, but it produces a message spew that certainly resembles an I/O error. Versions: |
At the moment every time a GRIB file is opened cfgrib needs to scan all the messages in the file to build the index that is then used to compute the values of the coordinates and build the hypercube representation of the variables.
Worse, when opening a GRIB file with the convenience function
open_datasets
the index is discarded every time the recursive call fails and the expensive file scan is done again.Proposed implementation requirements for the feature are:
path
+.idx
immediately after computationpath
+.idx
index file, test that it is in sync with the GRIB file and load itThe text was updated successfully, but these errors were encountered: