what happens to empty catalogs

[lmizrahi]

as discussed in the workshop, it seems like catalogs which contain zero events are not considered in the forecasts. empty catalogs are important information for a forecast. is there a way to ensure the correct number of empty catalogs are being used?

[wsavran]

noting that this is for the CatalogForecast.load_ascii_catalogs() implementation

[wsavran]

linking #90

[wsavran]

note for later: The follow snippet

import csep

fore = csep.load_catalog_forecast('ucerf3-landers_1992-06-28T11-57-34-14.csv')
# load catalogs lazily
for cat in fore:
    continue 

Throws the following error:

Traceback (most recent call last):
  File "/Users/wsavran/Applications/anaconda3/envs/test3/lib/python3.8/site-packages/csep/core/forecasts.py", line 553, in __next__
    n_items = len(self.catalogs)
TypeError: object of type 'generator' has no len()

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/wsavran/Applications/anaconda3/envs/test3/lib/python3.8/site-packages/csep/core/forecasts.py", line 565, in __next__
    catalog = next(self.catalogs)
  File "/Users/wsavran/Applications/anaconda3/envs/test3/lib/python3.8/site-packages/csep/core/catalogs.py", line 952, in load_ascii_catalogs
    lon = float(line[0])
ValueError: could not convert string to float: ''

ucerf3-landers_1992-06-28T11-57-34-14.csv.zip

Proposed solution: If no events in the catalog, the line should have the following format in the .csv file. Note this is about 14 bytes per catalog, so 1.4 Mb for a 100000 event catalog.

,,,,,9999,

Not completely optimized for storage, but this provides the most clear and straightforward implementation. Also, I think we should ensure the catalog forecast format is identical to the ascii catalog format except for the addition of the catalog_id field.

[wsavran]

opened pull request to fix this issue. now catalogs with no events can be explicitly specified in the file. an example forecast with 10 catalogs and no events in each catalog could be written as

,,,,,9,

this implies that there are 10 catalogs in the forecast and 0 events in each catalog. this is equivalent to the following representation.

,,,,,0,
,,,,,1,
,,,,,2,
,,,,,3,
,,,,,4,
,,,,,5,
,,,,,6,
,,,,,7,
,,,,,8,
,,,,,9,

these can be used in conjunction to create small forecast files as follows:

1,1,1,1992-01-01T0:0:0.0,1,3,1
,,,,,9,

implies that the forecast has 10 catalogs and 1 event in catalog_id=3. the most compact representation of this catalog would be

1,1,1,1992-01-01T0:0:0.0,1,9,1

more will be included in the documentation.