DL1 EventSource for ctapipe

[vuillaut]

Hi

I finally took the time to implement the DL1 EventSource for ctapipe.

I tested versus a simtel file that I converted into DL1.
When opened with ctapipe, both files give the same info.
With one exception: because we now merge the gains, we have only one channel for the charge and peakpos for all cameras.

Usage:

from dl1_data_handler.dl1dheventsource import DL1DHEventSource
source = DL1DHEventSource(input_url = filename)
for event in source:
...

I also created a separate module called ctapipe_io_dl1dh.
This allow ctapipe to automatically recognize it as a plugin once it is installed and thus allow for an even more simple and generic:

from ctapipe.io import event_source
source = event_source('dl1dh_file.h5')
for event in source:
    ....

I added the tests function but kept them commented as long as we don't have a test file.
At least you can use them to test and review ;-)

[vuillaut]

I did not include unit test as there is no test file in the package right now.
Shall we add such a test file?
Or I could create one during unit testing from the gamma_test.simtel.gz in ctapipe_resources?

[nietootein]

Hi @vuillaut. Thanks so much for this PR!

Did you try to convert gamma_test.simtel.gz into DL1? I encountered some issues when trying to read from some former prod3 files likely caused by an outdated eventio version in ctapipe (#52). If the file processes right I'd suggest to consider it for unit testing for now. Thoughts?

[vuillaut]

Did you try to convert gamma_test.simtel.gz into DL1? I encountered some issues when trying to read from some former prod3 files likely caused by an outdated eventio version in ctapipe (#52). If the file processes right I'd suggest to consider it for unit testing for now. Thoughts?

I tried in the meantime and you are right, it fails.

[nietootein]

Shall we try to upgrade to eventio 0.16.1 as @maxnoe suggested in #52, so can can start implementing unit testing right away?

[vuillaut]

Shall we try to upgrade to eventio 0.16.1 as @maxnoe suggested in #52, so can can start implementing unit testing right away?

dl1_data_handler is based on ctapipe, not on a specific simtel reader.
We'll get pyeventio with the next ctapipe release (along with the simtel info we are missing).

Or we use ctapipe master (but I'd not recommend, it's moving fast right now with major refactoring).
Maybe in a parallel development branch to see if we can get a free lunch without many changes.

[nietootein]

Then my personal opinion would be to wait for the next ctapipe release, since there are more urgent things to tend to right now...

[vuillaut]

Then this is ready for review I suppose :)

[vuillaut]

Ok sorry for the multiple commits and merge.
I went all the way and created a separate module called ctapipe_io_dl1dh with the reading function.
This allow ctapipe to automatically recognize it as a plugin once it is installed and thus allow for an even more simple and generic

from ctapipe.io import event_source
source = event_source('dl1dh_file.h5')
for event in source:
    ....

I added the tests function but kept them commented as long as we don't have a test file.
At least you can use them to test and review ;-)

[vuillaut]

Anyone for a review?

[nietootein]

In my to do list, hope to have time in the coming days... If very urgent, we can perhaps try to find someone else. Sorry!

[vuillaut]

In my to do list, hope to have time in the coming days... If very urgent, we can perhaps try to find someone else. Sorry!

No it's fine, I am using it from my branch so no pressure.
It was just to know if people were looking at it or not ;-)
Thanks.