float32 vs. float64

[falkben]

before rearranging the steps (summing then doing the corrections) we were getting a resulting "final" image of float64 (numpy dtype of >f8).

However, after rearranging, where we apply corrections on the individual images and then sum, the "final" image is float32 (>f4). Am a bit unsure as to why.

The attached file contains the data type of each frame in each .img file in the 2007 folder when running the full pipeline in the old version (summing first): https://github.com/spacetelescope/DRESSCode/tree/cdc02bce474e533f014269fb06e38197f93181a5.

From this, we can see that the float64 conversion occurs in the coincidence loss correction step (writing out the _coi.img files. After this -- all following computations remain in float64

main_dtypes.csv

In the rearranged step order where we apply corrections on individual observations first and then sum (https://github.com/spacetelescope/DRESSCode/tree/44c16db46267f65c7fff28a6d7a1ca9c426c988f), the coincidence loss correction step still results in float64, but subsequent calculations somehow go back to float32. Perhaps a result of using data in memory in between steps?

rearrange_dtypes.csv

[falkben]

I suspect that the original way, where we stored things in a datacube, required the entire datacube to have the same datatype, which increased it to float64, even for the primary image, which doesn't need that level of resolution.

In the new way, we save the different parts out separately, not in a datacube.

However, I'm still not clear why even the uncertainties, which are calculated as float64, end up summed as float32.

perhaps the summing only supports float32.