Hi !
I’d like to propose an improvement / change regarding EOReader’s use of temporary files.
Context
EOReader writes temporary files after loading bands, in order to:
- cache processed data
- speed up subsequent reads
This is very useful in many workflows and probably responds perfectly to the initial use cases !
Problem
In some use cases, this caching mechanism is unnecessary and introduces overhead.
Example workflow:
- Load a product's bands once (fits in memory)
- Compute a spectral index
- Discard the data
In this case, bands are never reloaded, temporary files are never reused, disk I/O becomes pure overhead
Expected behavior
It would be useful to have an option to disable temporary file writing entirely, so that all processing stays in memory, no intermediate GeoTIFFs are written.
Proposed solution
Introduce a parameter such as:
product.load(..., use_cache=False)
# OR
product.load(..., write_tmp=False)
I understand that a significant part of EOReader’s internal design may rely on this temporary file mechanism. However, this would be a very useful feature for my use case, which involves processing large volumes (many thousands) of cropped Sentinel/Landsat products that fit in memory, whereminimizing I/O and optimizing processing time is a key concern.
Would this fit EOReader’s design philosophy?
Thanks in advance for your feedback!
Hi !
I’d like to propose an improvement / change regarding EOReader’s use of temporary files.
Context
EOReader writes temporary files after loading bands, in order to:
This is very useful in many workflows and probably responds perfectly to the initial use cases !
Problem
In some use cases, this caching mechanism is unnecessary and introduces overhead.
Example workflow:
In this case, bands are never reloaded, temporary files are never reused, disk I/O becomes pure overhead
Expected behavior
It would be useful to have an option to disable temporary file writing entirely, so that all processing stays in memory, no intermediate GeoTIFFs are written.
Proposed solution
Introduce a parameter such as:
I understand that a significant part of EOReader’s internal design may rely on this temporary file mechanism. However, this would be a very useful feature for my use case, which involves processing large volumes (many thousands) of cropped Sentinel/Landsat products that fit in memory, whereminimizing I/O and optimizing processing time is a key concern.
Would this fit EOReader’s design philosophy?
Thanks in advance for your feedback!