Fix dtype promotion in mersi2_l1b reader
#2976
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #2976 +/- ##
=======================================
Coverage 96.10% 96.10%
=======================================
Files 377 377
Lines 55134 55147 +13
=======================================
+ Hits 52984 52997 +13
Misses 2150 2150
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Pull Request Test Coverage Report for Build 11792626537Details
💛 - Coveralls |
| @@ -512,6 +523,9 @@ def test_all_resolutions(self): | |||
| _test_multi_resolutions(available_datasets, self.ir_250_bands, resolution, ir_num_results) | |||
|
|
|||
| res = reader.load(self.bands_1000 + self.bands_250) | |||
| for i in res: | |||
| assert res[i].dtype == np.float32 | |||
There was a problem hiding this comment.
Need to compute the dask array for this too, right?
There was a problem hiding this comment.
Hah, I already had that ready on my work laptop but didn't push it. Added in 85f85af
| "Slope": np.array([.01] * 1, dtype=np.float32), | ||
| "Intercept": np.array([0.] * 1, dtype=np.float32), |
There was a problem hiding this comment.
Angles should remain in float64 right?
There was a problem hiding this comment.
Actually the SensorZenith data are H5T_STD_I16LE in the files. Slope and intercept are float32. Fixed in
fd2cec6
Btw, why is this also in the _get_1km_data(), while in the data it is not in the channel data files?
| @@ -237,7 +248,7 @@ def _get_geo_data(num_scans, rows_per_scan, num_cols, prefix): | |||
| xr.DataArray( | |||
| da.ones((num_scans * rows_per_scan, num_cols), chunks=1024), | |||
| attrs={ | |||
| "Slope": np.array([1.] * 1), "Intercept": np.array([0.] * 1), | |||
| "Slope": np.array([1.] * 1, dtype=np.float32), "Intercept": np.array([0.] * 1, dtype=np.float32), | |||
There was a problem hiding this comment.
Angles should remain in float64 right?
There was a problem hiding this comment.
Yes, Longitude and its intercept and slope seem to be float64. Fixed in 3bb6db5
| @@ -246,7 +257,7 @@ def _get_geo_data(num_scans, rows_per_scan, num_cols, prefix): | |||
| xr.DataArray( | |||
| da.ones((num_scans * rows_per_scan, num_cols), chunks=1024), | |||
| attrs={ | |||
| "Slope": np.array([1.] * 1), "Intercept": np.array([0.] * 1), | |||
| "Slope": np.array([1.] * 1, dtype=np.float32), "Intercept": np.array([0.] * 1, dtype=np.float32), | |||
There was a problem hiding this comment.
Angles should remain in float64 right?
There was a problem hiding this comment.
Yes, Latitude and its intercept and slope seem to be float64. Fixed in 3bb6db5
| @@ -255,7 +266,7 @@ def _get_geo_data(num_scans, rows_per_scan, num_cols, prefix): | |||
| xr.DataArray( | |||
| da.ones((num_scans * rows_per_scan, num_cols), chunks=1024), | |||
| attrs={ | |||
| "Slope": np.array([.01] * 1), "Intercept": np.array([0.] * 1), | |||
| "Slope": np.array([.01] * 1, dtype=np.float32), "Intercept": np.array([0.] * 1, dtype=np.float32), | |||
There was a problem hiding this comment.
Angles should remain in float64 right?
There was a problem hiding this comment.
Actually the SensorZenith data are H5T_STD_I16LE in the files. Slope and intercept are float32. Fixed in 3bb6db5
With Numpy 2 the
dtypeof read data firmersi2_l1breader changed fromfloat32tofloat64. The fix was simple, but I'm not entirely sure the changes in the tests reflect the actual data.The memory usage for my test composites went from 14 GB down to 10 GB.