Ra-Dec #253

Elisa-Visentin · 2024-09-06T10:04:01Z

Add ra-dec and MC dec information in LST database

codecov · 2024-09-06T10:15:18Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.23%. Comparing base (cea9fa6) to head (a3387b3).
Report is 5 commits behind head on auto_MCP_DL2_DL3.

Additional details and impacted files

@@                Coverage Diff                @@
##           auto_MCP_DL2_DL3     #253   +/-   ##
=================================================
  Coverage             77.23%   77.23%           
=================================================
  Files                    21       21           
  Lines                  2614     2614           
=================================================
  Hits                   2019     2019           
  Misses                  595      595

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Elisa-Visentin · 2024-09-09T08:34:12Z

This new scripts seems to work fine. We still miss some entries into the dictionaries, but we just have to update the txt files later on (while working on Dl2 and DL3): ra, dec, MC_dec and point_source cols. are added to the LST database

aleberti · 2024-09-09T14:28:21Z

Why do you need three files to specify the RA, Dec and point-like/extended property of the same sources? Can't you have e.g. a single yaml file to store all that info? Or also JSON, since you use it anyway to open the txt files.

Elisa-Visentin · 2024-09-10T06:24:10Z

Why do you need three files to specify the RA, Dec and point-like/extended property of the same sources? Can't you have e.g. a single yaml file to store all that info? Or also JSON, since you use it anyway to open the txt files.

A single file with 3 dictionaries inside? Maybe quite a mess when we have to modify it (hundreds of lines and quite difficult to find out if a source is already there in which dictionary)
We could store each entry as a list ('source':[ra, dec, point]) but then some of the entries of the list will be null (not known or not needed): what about this solution?

Elisa-Visentin · 2024-09-10T06:25:42Z

We also have a problem with Crab (during one of the obs. they used 'Crab' instead of 'CrabNebula' and astropy returns two different coordinates, while I think that they should have the same coords.

aleberti · 2024-09-10T06:36:31Z

Yes, I meant the solution you proposed i.e. one list for each source, and each source name becomes a key.

Regarding the Crab/CrabNebula, it is likely our problem to solve. 99% I am sure that the configuration on the LST side have the same coordonates. Astropy gives slightly different ones because most probably Crab is the pulsar (not sure which catalog is used though).

Elisa-Visentin · 2024-09-10T06:42:37Z

I'm almost sure that the source is always the same (as for the observations): probably they just put the wrong name. astropy retrieves coordinates from simbad usually. This can be easily solved in this script by adding an 'if', but I will also check MAGIC root files for one of these obs (just to be sure that the coordinates are the same)

aleberti · 2024-09-10T08:37:46Z

We also have a problem with Crab (during one of the obs. they used 'Crab' instead of 'CrabNebula' and astropy returns two different coordinates, while I think that they should have the same coords.

here are we talking about MAGIC or LST? in LST I see only one configuration called crab (plus other configurations with different wobble offset, but with very explicative names). Of course I cannot exclude that there was another configuration with a different name that was deleted and therefore not listed anymore.

Elisa-Visentin · 2024-09-10T09:05:35Z

Now I am checking MAGIC coordinates (easier: they are written in the files) and there seem to exist two different sources (Crab and CrabNebula) according to MARS and data taking: also in the runbook there is a different name in schedule wrt run summary

aleberti · 2024-09-10T09:09:34Z

@Elisa-Visentin can you tell one of these dates? I would like to check better, because in the scheduling catalog files I did not find source with the name "Crab", only "CrabNebula".

Elisa-Visentin · 2024-09-10T09:37:52Z

OK, from MAGIC:
CrabNebula: 22° 00′ 52″, 05:34:32 (h:m:s) deg: 22.0144444 83.6333333

Crab: 22° 00′ 51", 05:34:31 (h:m:s) deg: 22.0141667 83.6291667

mhhh... the second one does not even correspond to the one by astropy

Elisa-Visentin · 2024-09-10T09:41:35Z

3-4 March 2024 (LST): Yes the name is the 'wrong' one in run summary, not in scheduling

Elisa-Visentin · 2024-09-10T09:45:56Z

In the mean time, I fixed the input dictionary

aleberti · 2024-09-10T09:55:34Z

ok, apparently there is a catalog of sources in MAGIC called "favorites" where indeed we have:

Crab,f|T,+5:34:31.00,+22:0:51.00,1.0,2000

that's what shifters used.

Elisa-Visentin · 2024-09-10T09:58:48Z

Ok so now:

we ignore the sec. difference and put the same coordinates (which?) for both of them to be able to stack data?
we keep the two sources as different and fix the Crab coordinates to the ones actually used during the obs.?

aleberti · 2024-09-10T10:07:34Z

we ignore the sec. difference and put the same coordinates (which?) for both of them to be able to stack data?

effectively for observations that difference does not affect analysis. Please use the coordinates of CrabNebula

we keep the two sources as different and fix the Crab coordinates to the ones actually used during the obs.?

I already modified the only catalog file of MAGIC where the name was Crab instead of CrabNebula. So now it is:

CrabNebula,f|T,5:34:32.00,22:00:52.00,1.0,2000

so, it was just really a remote coincidence that shifters used that catalog...

Elisa-Visentin · 2024-09-10T10:16:49Z

fixed. Thanks

jsitarek · 2024-09-12T10:57:26Z

magicctapipe/scripts/lst1_magic/semi_automatic_scripts/database_production/set_ra_dec.py

+Add, to LST database, infos about coordinates and extension of the sources and MC declination to be used to process the source
+
+Usage:
+$ set_ra_dec (-b YYYYMMDD -e YYYYMMDD -r ra_dict -d dec_dict -m mc_dec -p point_source_dict)


options -r -d and -p were changes into one -s

jsitarek · 2024-09-12T11:04:49Z

magicctapipe/scripts/lst1_magic/semi_automatic_scripts/database_production/set_ra_dec.py

+                coord = SkyCoord.from_name("CrabNebula")
+            src_dec = coord.dec.degree
+            src_ra = coord.ra.degree
+            df_LST["ra"] = np.where(df_LST["source"] == src, src_ra, df_LST["ra"])


those 3 lines are also repeated in the except block of the code. I think you can put them once after the except block always setting them to src_ra, src_dec and min dec if those are available, and if not to Nan

jsitarek · 2024-09-12T11:09:17Z

magicctapipe/scripts/lst1_magic/semi_automatic_scripts/database_production/set_ra_dec.py

+            )
+        except NameResolveError:
+            print(f"{i}: {src} not found in astropy. Looking to the dictionaries...")
+            if (src in source_dict.keys()) and (source_dict.get(src)[0] != "NaN"):


maybe you can read RA and Dec together? they should be both either available or both NaN, and it would simplify the code

jsitarek

I put a few small comments

Elisa-Visentin changed the title ~~first dict and lists~~ Ra-Dec Sep 6, 2024

Elisa-Visentin marked this pull request as ready for review September 9, 2024 08:32

Elisa-Visentin requested review from aleberti, jsitarek and joannaWojS September 9, 2024 08:33

aleberti approved these changes Sep 10, 2024

View reviewed changes

jsitarek reviewed Sep 12, 2024

View reviewed changes

jsitarek requested changes Sep 12, 2024

View reviewed changes

Elisa-Visentin force-pushed the db_ra_dec branch from a3387b3 to 808627a Compare September 12, 2024 12:54

Elisa-Visentin closed this Sep 12, 2024

Elisa-Visentin force-pushed the db_ra_dec branch from 808627a to cea9fa6 Compare September 12, 2024 12:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ra-Dec #253

Ra-Dec #253

Elisa-Visentin commented Sep 6, 2024

codecov bot commented Sep 6, 2024 •

edited

Loading

Elisa-Visentin commented Sep 9, 2024 •

edited

Loading

aleberti commented Sep 9, 2024 •

edited

Loading

Elisa-Visentin commented Sep 10, 2024

Elisa-Visentin commented Sep 10, 2024

aleberti commented Sep 10, 2024

Elisa-Visentin commented Sep 10, 2024

aleberti commented Sep 10, 2024

Elisa-Visentin commented Sep 10, 2024

aleberti commented Sep 10, 2024

Elisa-Visentin commented Sep 10, 2024

Elisa-Visentin commented Sep 10, 2024

Elisa-Visentin commented Sep 10, 2024

aleberti commented Sep 10, 2024

Elisa-Visentin commented Sep 10, 2024

aleberti commented Sep 10, 2024

Elisa-Visentin commented Sep 10, 2024

jsitarek Sep 12, 2024 •

edited

Loading

jsitarek Sep 12, 2024

jsitarek Sep 12, 2024

jsitarek left a comment

Ra-Dec #253

Ra-Dec #253

Conversation

Elisa-Visentin commented Sep 6, 2024

codecov bot commented Sep 6, 2024 • edited Loading

Codecov Report

Elisa-Visentin commented Sep 9, 2024 • edited Loading

aleberti commented Sep 9, 2024 • edited Loading

Elisa-Visentin commented Sep 10, 2024

Elisa-Visentin commented Sep 10, 2024

aleberti commented Sep 10, 2024

Elisa-Visentin commented Sep 10, 2024

aleberti commented Sep 10, 2024

Elisa-Visentin commented Sep 10, 2024

aleberti commented Sep 10, 2024

Elisa-Visentin commented Sep 10, 2024

Elisa-Visentin commented Sep 10, 2024

Elisa-Visentin commented Sep 10, 2024

aleberti commented Sep 10, 2024

Elisa-Visentin commented Sep 10, 2024

aleberti commented Sep 10, 2024

Elisa-Visentin commented Sep 10, 2024

jsitarek Sep 12, 2024 • edited Loading

Choose a reason for hiding this comment

jsitarek Sep 12, 2024

Choose a reason for hiding this comment

jsitarek Sep 12, 2024

Choose a reason for hiding this comment

jsitarek left a comment

Choose a reason for hiding this comment

codecov bot commented Sep 6, 2024 •

edited

Loading

Elisa-Visentin commented Sep 9, 2024 •

edited

Loading

aleberti commented Sep 9, 2024 •

edited

Loading

jsitarek Sep 12, 2024 •

edited

Loading