Mirage can extract a bad "zero frame" for sub-array darks.

[KevinVolkSTScI]

I have run into an error when using a sub-array dark as input to Mirage for creating a simulation. The reason I need to use a sub-array dark is that the observation needs 800 frames, and an 800 frame full array dark is too large for python to handle due to memory constraints. In this particular case I am using a 64x64 pixel subarray that is in the upper right corner of the array hence it has a description in the full array of [1983:, 1983:]. The issue happens in dark_prep.py where it has a statement

                model.zeroframe = model.zeroframe[:, self.subarray_bounds[1]:
                                                  self.subarray_bounds[3] + 1,
                                                  self.subarray_bounds[0]:self.subarray_bounds[2] + 1]

in line 224 of the Mirage version I was looking at. The self.array_bounds values read from the header are the position of the sub-array in the full frame in x and in y, but the input model.zeroframe has dimension [64, 64] coming into this statement so when it tries to extract [1983:2048, 1983:2048] an empty array is returned. (Not 'None', an array of dimension [0, 0].) Eventually this cause the code to crash with an error message such as

Averaging frame 23 into group 23
Averaging frame 24 into group 24

Traceback (most recent call last):
File "./runmodels.py", line 64, in
t1.create()
File "/home/kvolk/anaconda3/envs/mirage_30march2021/lib/python3.7/site-packages/mirage/imaging_simulator.py", line 82, in create
obs.create()
File "/home/kvolk/anaconda3/envs/mirage_30march2021/lib/python3.7/site-packages/mirage/ramp_generator/obs_generator.py", line 1232, in create
syn_zeroframe=simzero)
File "/home/kvolk/anaconda3/envs/mirage_30march2021/lib/python3.7/site-packages/mirage/ramp_generator/obs_generator.py", line 549, in add_synthetic_to_dark
zeroframe = dark.zeroframe + syn_zeroframe
ValueError: operands could not be broadcast together with shapes (5,0,0) (5,64,64)

In this case the simulation has 5 exposures in the integration so the output of the zeroframe extraction is copied 5 times to make the dimension [5, 0, 0] which then the code tries to add to the simulated zero frames which have dimension [5, 64, 64].

If by chance the sub-array starts at [1, 1]--in IRAF-like notation--then the code will not crash this way, but any other lower left corner position for the sub-array will have this type of problem as far as I can see.

The fix would seem to be to check whether the dark.zeroframe is of dimension [2048, 2048] before doing the extraction in this way, and otherwise just taking the zeroframe as it is (possibly checking that the dimensions are as expected...).

There are several similar statements to the one shown above in the dark_prep.py code, all of which need a fix it seems.

[KevinVolkSTScI]

I will attempt to code a fix for this tomorrow. I am just a bit worried that I do not understand all the ins and outs of what the dark object is doing, so I do not want to jump in blind and start changing the code.

[KevinVolkSTScI]

If I change the code to add a test for the size of the sub-array being extracted, right at the start of routine crop-dark in file dark_prep.py (line 207 in the version I am looking at) as follows:

    modshape = model.data.shape
    yd = modshape[-2]
    xd = modshape[-1]
    xs = self.subarray_bounds[2] - self.subarray_bounds[0] + 1
    ys = self.subarray_bounds[3] - self.subarray_bounds[1] + 1

    if (xd > xs) and (yd > ys) and (xd > self.subarray_bounds[2]) and \
        (yd > self.subarray_bounds[3]):
        if ((self.subarray_bounds[0] != 0) or (self.subarray_bounds[2] != (xd - 1))
           or (self.subarray_bounds[1] != 0) or (self.subarray_bounds[3] != (yd - 1))):

            if len(modshape) == 4:
                model.data = model.data[:, :, self.subarray_bounds[1]:self.subarray_bounds[3] + 1,
                                        self.subarray_bounds[0]:self.subarray_bounds[2] + 1]
                try:
                    model.sbAndRefpix = model.sbAndRefpix[:, :, self.subarray_bounds[1]:
                                                          self.subarray_bounds[3] + 1,
                                                          self.subarray_bounds[0]:self.subarray_bounds[2] + 1]
                except:
                    pass
                try:
                    model.zeroframe = model.zeroframe[:, self.subarray_bounds[1]:
                                                      self.subarray_bounds[3] + 1,
                                                      self.subarray_bounds[0]:self.subarray_bounds[2] + 1]
                except:
                    pass
            if len(modshape) == 3:
                model.data = model.data[:, self.subarray_bounds[1]:self.subarray_bounds[3] + 1,
                                        self.subarray_bounds[0]:self.subarray_bounds[2] + 1]
                try:
                    model.sbAndRefpix = model.sbAndRefpix[:, self.subarray_bounds[1]:
                                                          self.subarray_bounds[3] + 1,
                                                          self.subarray_bounds[0]:self.subarray_bounds[2] + 1]
                except:
                    pass
                try:
                    model.zeroframe = model.zeroframe[:, self.subarray_bounds[1]:
                                                      self.subarray_bounds[3] + 1,
                                                      self.subarray_bounds[0]:self.subarray_bounds[2] + 1]
                except:
                    pass
            elif len(modshape) == 2:
                model.data = model.data[self.subarray_bounds[1]:self.subarray_bounds[3] + 1,
                                        self.subarray_bounds[0]:self.subarray_bounds[2] + 1]
                try:
                    model.sbAndRefpix = model.sbAndRefpix[self.subarray_bounds[1]:self.subarray_bounds[3] + 1,
                                                          self.subarray_bounds[0]:self.subarray_bounds[2] + 1]
                except:
                    pass
                try:
                    model.zeroframe = model.zeroframe[self.subarray_bounds[1]:
                                                      self.subarray_bounds[3] + 1,
                                                      self.subarray_bounds[0]:self.subarray_bounds[2] + 1]
                except:
                    pass

the code seems to work OK. I am not sure, however, whether this might cause issue elsewhere in the code in terms of the pedestal file that is being used if that is coming from a full frame reference file when the dark ramp is a sub-array dark ramp.