MemoryError when querying GEDI data with GEDIdb

[MabelSSantos]

Hi, everyone!

I’m working with the gedidb, and I’m running into a MemoryError when trying to query GEDI data. My goal is to extract biomass variables (agbd, agbd_pi_lower, agbd_pi_upper, etc.) and I have created a grid to extract data for a large area. Even after repeatedly decreasing the grid tile size and narrowing the time range, I keep getting the MemoryError.
I am now at a square grid size of 0.05 degrees and only one year range, which should not create a memory problem, but keep getting the error message.

Here is my query code:

-> Define variables to query and quality filters
vars_selected = ["agbd", 'agbd_pi_lower', "agbd_pi_upper", "agbd_se", "l4_quality_flag", #biomass data
"shot_number", "beam_name"] #beam info

-> Define quality filters
quality_filters = {
'sensitivity': '>= 0.95 and <= 1.0',
'beam_type': "== 'full'",
'l2a_quality_flag' : '== 1',
'degrade_flag': '== 1',
'surface_flag': '==1'
}

-> Get the data
rsps = provider.get_data(
variables=vars_selected,
query_type="bounding_box",
geometry=sq0,
start_time="2019-04-01",
end_time="2019-12-31",
return_type='dataframe',
**quality_filters)

This is the error message:

2025-11-28 14:45:44,463 - ERROR - Error querying TileDB array 's3://dog.gedidb.gedi-l2-l4-v002/array_uri': MemoryError: Unable to allocate 16.0 GiB for an array with shape (2147483648,) and data type uint64 (D:\a\TileDB-Py\TileDB-Py\tiledb\core.cc:105)

Could this be caused by the way I’m passing filters? Is the error due to TileDB trying to load too much data at once, even for a small bounding box? What’s the recommended way to chunk queries? Are there best practices for streaming or limiting GEDI queries to avoid memory overload?

I would greatly appreciate any help or suggestions that you could give me!

[simonbesnard1]

Hi @MabelSSantos

It seems to be the same issue as in #81. It's not an issue with gedidb but rather a working environment issue. Are you working in windows and WSL?

[MabelSSantos]

Hi, Simon! Thank you for your fast answer! Yes, I'm working on Windows, but using PowerShell rather than WSL. I read your response in #81, but I'm not sure how to apply those suggestions in my setup. Do you have any suggestions on how I should proceed in this case, or what adjustments I need to make so your solution (using export TILEDB_FORCE_RANGE_COORDS=1 ) works for me?

[simonbesnard1]

Can you use gedidb in WSL?