Allow pipeline to process QHY600 CMOS detectors #132

abostroem · 2024-02-23T23:05:04Z

This PR was modeled off of PR #80 which implemented MuSCAT

Other changes:

This also slightly changes the way the color correction is used when no instrument is found. Previously there were site colors defined. I have removed them and now it defaults to 0 color correction and prints a warning to the screen that the color correction is 0 and suggests using the --unfix option to calculate a color correction.
Fixed a bug in the wcs calculation (only used when BANZAI fails) for the fwhmgess3 for the sinistro telescopes

Testing:
I tested this on the 2024-02-06 data from proposal LCOEPO2014B-010 of SN 2024bch

I ran the following commands:

lscloop.py -n 2024bch -e 20240206 -T sq -s psf
lscloop.py -n 2024bch -e 20240206 -T sq -s psfmag
lscloop.py -n 2024bch -e 20240206 -T sq -s zcat #This results in no color correction applied
lscloop.py -n 2024bch -e 20240206 -T sq -s zcat --unfix -F #this calculates the color correction
lscloop.py -n 2024bch -e 20240206 -T sq -s mag

I compared the final photometry for a few different epochs and ii was all pretty close to the 1m reduction taken at about the same time.

To test the wcs stage:
lscloop.py -n 2024bch -e 20240206 -T sq --id 184 -s wcs

I then loaded the BANZAI and wcs stage image into DS9 and matched WCS - they aligned pretty closely

I noticed a weird bug here that the WCS gets updated but the other database terms aren't reset because the filename contains an extra .fits. But this isn't a bug introduced by this PR and it doesn't break anything, so I'm not going to fix it right now. I've filed this as Issue 131.

Note that this PR does not check difference imaging capabilities

…recommend users use the --unfixed option if no color term exists for their instrument

abostroem · 2024-02-23T23:05:36Z

trunk/src/lsc/lscastrodef.py

            elif 'fl' in _instrume or 'fa' in _instrume:
                fwhmgess3=half_total_flux_radius_to_fwhm(median(array(fwhm3))) * 0.389
-                if _imex:  fwhmgessime = median(array(ccc)) * 0.467
+                if _imex:  fwhmgessime = median(array(ccc)) * 0.389


abostroem · 2024-02-23T23:06:14Z

Note the 0.734 pixel scale value comes from the file header

abostroem · 2024-02-23T23:10:52Z

I'll also note I was able to download and ingest these files with LCOGTingest.py without making any changes anywhere

griffin-h

Looks good, just left some minor comments. I assume @cmccully will actually test this on the supernova machine, but let me know if you need me to do it.

griffin-h · 2024-02-23T23:16:17Z

trunk/src/lsc/util.py

@@ -138,6 +138,31 @@ def readkey3(hdr,keyword):
    except: 
       _instrume='none'
    if 'kb' in _instrume: # SBIG


Can this just be if 'kb' in _instrume or 'sq' in _instrume:? Pre-2014, different instruments had different header keywords, but they have been standard since then.

fixed with recent push

griffin-h · 2024-02-23T23:20:25Z

trunk/src/lsc/mysqldef.py

    elif prefix == 'ep':
        insttype = 'MUSCAT'
+    elif prefix == 'sq':
+        insttype = 'qhy'


Some of these instruments are already in the SNEx database with the type QHY (caps). Ignoring the fact that I have no idea where that came from, let's make it consistent here.

On the other hand, there are already both MuSCAT and MUSCAT in the database, so maybe this doesn't matter.

I can make it consistent - even if SQL doesn't care

fixed with recent push

griffin-h · 2024-02-23T23:24:37Z

trunk/src/lsc/lscastrodef.py

+            if hdr['SITEID'] in ['elp','tfn','ogg']:
+                CD1_1 = (-1) * CD1_1
+                CD2_2 = (-1) * CD2_2


Is this actually true? It looks like this came from copying the Sinistro block, but I have no idea if the same random sites have 180º rotated cameras.

Stefano said this is northern vs southern hemisphere

I actually have no idea. Probably Griffin is right is just a rotation of the camera.

I didn't test the southern hemisphere but it did something sensible for tfn when I ran the wcs stage

trunk/bin/lscloop.py

abostroem · 2024-03-20T16:58:15Z

What has to happen to get this PR approved? Now that the QHY600 detectors are available in SNEx, I'd like to get it merged. I think if we were doing the WCS wrong for one of the hemispheres it would have come up by now, all other comments have been addressed

Simplification to guessing the starting WCS for all cameras.

abostroem · 2024-03-28T20:44:43Z

I reran the commands described above with PR #134 merged in - everything looks great. @estefaniapadilla can you please run these commands on supernova (note this is EPO data, so we might not be automatically downloading it - alternately, you could run it on a standard e.g. L107 which we might be downloading automatically)

abostroem · 2024-03-28T20:45:37Z

@griffin-h can you approve the changes @cmccully and I made?

griffin-h

Looks fine to me. I didn't test the new WCS code but I trust that Curtis did something reasonable. All my other comments are resolved.

abostroem · 2024-04-04T22:34:04Z

I tested the new WCS code in PR #134

griffin-h · 2024-04-04T22:43:30Z

I tested the new WCS code in PR #134

Got it, I didn't realize that #134 was a PR into this PR, and not into master.

estefaniapadilla · 2024-04-11T22:50:26Z

tested the commands on the supernova machine and worked as expected

svalenti and others added 4 commits February 23, 2024 20:46

adding qhy (cmos)

3040cef

updating color terms to never define color correction by site and to …

42b5df5

…recommend users use the --unfixed option if no color term exists for their instrument

updating manual with qhy/sq instrument

cd09ded

adding fwhmgess3 for sq instrument

61c147b

abostroem requested review from cmccully and griffin-h February 23, 2024 23:05

abostroem commented Feb 23, 2024

View reviewed changes

adding qhy to instrument options in documentation

c894d0c

griffin-h requested changes Feb 23, 2024

View reviewed changes

abostroem added 3 commits February 23, 2024 23:41

condensing instrument definitions

5232520

changing qhy to QHY

c9569bc

fixing typo

ec9dc2b

abostroem mentioned this pull request Mar 22, 2024

Correcting for a missing gain factor in the noise image creation #133

Open

cmccully and others added 2 commits March 27, 2024 13:30

Simplification to guessing the starting WCS for all cameras.

369ae1d

Merge pull request #134 from LCOGT/wcs-guess

2fe2374

Simplification to guessing the starting WCS for all cameras.

abostroem requested a review from estefaniapadilla March 28, 2024 20:45

abostroem requested a review from griffin-h March 28, 2024 20:54

griffin-h approved these changes Apr 4, 2024

View reviewed changes

estefaniapadilla merged commit cae5a6a into master Apr 11, 2024

Allow pipeline to process QHY600 CMOS detectors #132

Allow pipeline to process QHY600 CMOS detectors #132

Uh oh!

Conversation

abostroem commented Feb 23, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

abostroem commented Feb 23, 2024

Uh oh!

abostroem commented Feb 23, 2024

Uh oh!

griffin-h left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

abostroem commented Mar 20, 2024

Uh oh!

abostroem commented Mar 28, 2024

Uh oh!

abostroem commented Mar 28, 2024

Uh oh!

griffin-h left a comment

Choose a reason for hiding this comment

Uh oh!

abostroem commented Apr 4, 2024

Uh oh!

griffin-h commented Apr 4, 2024

Uh oh!

estefaniapadilla commented Apr 11, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants