Issue with multiple prices, "View trading hours for all instruments" crashes

[j1404]

I'm stuck with multiple prices, any advice is greatly appreciated. Originally I had issue with some orders not being executed, which was a problem with time zones and is now fixed. During the troubleshooting I noticed that "View trading hours for all instruments" crashes because of some issue with multiple prices:

Your choice? <RETURN for EXIT> 1
10: View instrument configuration data
11: View contract configuration data
12: View trading hours for all instruments


Your choice? <RETURN for Back> 12
2024-01-17 21:39:03 DEBUG config {'type': 'config', 'stage': 'config'} Adding config defaults
2024-01-17 21:39:03 DEBUG root Cache has expired data, fetching from slow path and reloading cache.
 [--------------------------------------------------------------------------------] 0.0% Progress (0.0/0.0/0.0 secs left/elapsed/total)2024-01-17 21:39:04 DEBUG Interactive-Diagnostics {'component': 'mongoIbBrokerClientIdData'} Locked IB client ID 200
 [========-----------------------------------------------------------------------] 10.4% Progress (108.9/18.0/126.9 secs left/elapsed/total [=========----------------------------------------------------------------------] 11.6% Progress (111.4/19.7/131.1 secs left/elapsed/total [==========---------------------------------------------------------------------] 12.7% Progress (110.2/21.2/131.4 secs left/elapsed/total [===========--------------------------------------------------------------------] 13.9% Progress (110.1/22.7/132.8 secs left/elapsed/total [============-------------------------------------------------------------------] 15.0% Progress (116.0/24.7/140.7 secs left/elapsed/total [=============------------------------------------------------------------------] 16.8% Progress (108.2/26.7/134.9 secs left/elapsed/total [==============-----------------------------------------------------------------] 17.9% Progress (100.5/27.7/128.3 secs left/elapsed/total [===================------------------------------------------------------------] 24.3% Progress (100.5/36.3/136.8 secs left/elapsed/total [====================-----------------------------------------------------------] 25.4% Progress (100.1/37.9/138.0 secs left/elapsed/total [=====================================------------------------------------------] 46.8% Progress (50.7/60.3/111.0 secs left/elapsed/total)2024-01-17 21:40:04 DEBUG Interactive-Diagnostics {'broker': 'IB', 'clientid': 200} Terminating {'ipaddress': 'localhost', 'port': 4001, 'client': 200}
2024-01-17 21:40:04 DEBUG Interactive-Diagnostics {'component': 'mongoIbBrokerClientIdData'} Released IB client ID 200
Traceback (most recent call last):
  File "/home/alpha/pysystemtrade/sysproduction/linux/scripts/run.py", line 67, in <module>
    func(*args, **kwargs)
  File "/home/alpha/pysystemtrade/sysproduction/interactive_diagnostics.py", line 78, in interactive_diagnostics
    menu.run_menu()
  File "/home/alpha/pysystemtrade/syscore/interactive/menus.py", line 64, in run_menu
    method_chosen(*self._args, **self._kwargs)
  File "/home/alpha/pysystemtrade/sysproduction/interactive_diagnostics.py", line 605, in print_trading_hours_for_all_instruments
    all_trading_hours = get_trading_hours_for_all_instruments(data)
  File "/home/alpha/pysystemtrade/sysproduction/interactive_diagnostics.py", line 666, in get_trading_hours_for_all_instruments
    trading_hours = get_trading_hours_for_instrument(data, instrument_code)
  File "/home/alpha/pysystemtrade/sysproduction/interactive_diagnostics.py", line 703, in get_trading_hours_for_instrument
    contract_id = diag_contracts.get_priced_contract_id(instrument_code)
  File "/home/alpha/pysystemtrade/sysproduction/data/contracts.py", line 216, in get_priced_contract_id
    contract_dict = self.get_current_contract_dict(instrument_code)
  File "/home/alpha/pysystemtrade/sysproduction/data/contracts.py", line 166, in get_current_contract_dict
    current_contracts = multiple_prices.current_contract_dict()
  File "/home/alpha/pysystemtrade/sysobjects/multiple_prices.py", line 130, in current_contract_dict
    raise missingData
syscore.exceptions.missingData

Related to this, update_multiple_adjusted_prices complained the following for a bunch of instruments. These I just deleted for now, but that didn't solve the problem.

2024-01-17 19:27:37 CRITICAL Update-Multiple-Adjusted-Prices {'instrument_code': 'ZAR'} Something went wrong with adjusted prices for ZAR! [Can't update adjusted prices for ZAR as roll has occured but not registered properly]

Multiple prices for most of the instruments seem to have data in all columns, but then there are those like the following, where some of the carry or forward prices are missing.

Instrument code?AUD
                      CARRY CARRY_CONTRACT    PRICE PRICE_CONTRACT  FORWARD FORWARD_CONTRACT
index                                                                                       
1987-06-16 23:00:00  0.6972       19871200  0.70920       19870900   0.6972         19871200
1987-06-17 23:00:00  0.7042       19871200  0.71400       19870900   0.7042         19871200
1987-06-18 23:00:00  0.7027       19871200  0.71250       19870900   0.7027         19871200
1987-06-19 23:00:00  0.7020       19871200  0.71200       19870900   0.7020         19871200
1987-06-22 23:00:00  0.7013       19871200  0.71120       19870900   0.7013         19871200
...                     ...            ...      ...            ...      ...              ...
2024-01-16 22:00:00     NaN       20240900  0.66100       20240600      NaN         20240900
2024-01-16 23:00:00  0.6617       20240900  0.66100       20240600   0.6617         20240900
2024-01-17 16:30:00     NaN       20240900  0.65590       20240600      NaN         20240900
2024-01-17 17:00:00  0.6567       20240900  0.65665       20240600   0.6567         20240900
2024-01-17 18:00:00     NaN       20240900  0.65650       20240600      NaN         20240900


Instrument code?AUD_micro
                      CARRY CARRY_CONTRACT   PRICE PRICE_CONTRACT  FORWARD FORWARD_CONTRACT
index                                                                                      
2022-12-14 23:00:00     NaN       20230600  0.6873       20230300      NaN         20230600
2022-12-15 23:00:00     NaN       20230600  0.6727       20230300      NaN         20230600
2022-12-16 23:00:00     NaN       20230600  0.6716       20230300      NaN         20230600
2022-12-19 23:00:00     NaN       20230600  0.6717       20230300      NaN         20230600
2022-12-20 23:00:00  0.6725       20230600  0.6700       20230300   0.6725         20230600
...                     ...            ...     ...            ...      ...              ...
2024-01-16 23:00:00  0.6612       20240600  0.6597       20240300   0.6612         20240600
2024-01-17 16:30:00  0.6557       20240600  0.6541       20240300   0.6557         20240600
2024-01-17 17:00:00  0.6565       20240600  0.6549       20240300   0.6565         20240600
2024-01-17 18:00:00     NaN       20240600  0.6550       20240300      NaN         20240600
2024-01-17 19:00:00  0.6560       20240600  0.6545       20240300   0.6560         20240600


Instrument code?BB3M
                       CARRY CARRY_CONTRACT   PRICE PRICE_CONTRACT  FORWARD FORWARD_CONTRACT
index                                                                                       
2022-03-15 23:00:00  98.5675       20220600  98.255       20220900   97.905         20221200
2022-03-16 23:00:00  98.4950       20220600  98.145       20220900   97.765         20221200
2022-03-17 23:00:00  98.6100       20220600  98.175       20220900   97.755         20221200
2022-03-18 23:00:00  98.6050       20220600  98.145       20220900   97.710         20221200
2022-03-21 23:00:00  98.4800       20220600  97.930       20220900   97.495         20221200
...                      ...            ...     ...            ...      ...              ...
2024-01-09 23:00:00      NaN       20231200  94.775       20240300   95.225         20240600
2024-01-10 23:00:00      NaN       20231200  94.790       20240300   95.240         20240600
2024-01-11 23:00:00      NaN       20231200  94.820       20240300   95.300         20240600
2024-01-12 23:00:00      NaN       20231200  94.870       20240300   95.435         20240600
2024-01-16 23:00:00      NaN       20231200  94.805       20240300   95.370         20240600

Is there are way to find out on what instrument(s) the diagnostic script fails and/or how do I solve this?

[frankcodespoorly]

I had a similar problem with viewing trading hours while setting things up.

#1109 (reply in thread)

Did you complete the roll for ZAR? If the system had a position in ZAR that was rolled, after the roll executes, you need to use interactive_update_roll_status to update the adjusted price series

[robcarver17]

"but then there are those like the following, where some of the carry or forward prices are missing." - this is normal with illiquid instruments as the default is to only snapshot hourly prices if a trade occurs.

[craigmediaservices]

You can add an extra catch for the missing instrument data

def get_trading_hours_for_all_instruments(data=arg_not_supplied):
    if data is arg_not_supplied:
        data = dataBlob()

    diag_prices = diagPrices(data)
    list_of_instruments = diag_prices.get_list_of_instruments_with_contract_prices()

    p = progressBar(len(list_of_instruments))
    all_trading_hours = {}
    for instrument_code in list_of_instruments:
        print(f"Processing instrument: {instrument_code}")  # Log current instrument
        p.iterate()
        try:
            trading_hours = get_trading_hours_for_instrument(data, instrument_code)
        except missingContract:
            print(f"*** NO TRADING HOURS FOR {instrument_code} ***")
            continue
        except missingData:
            print(f"*** MISSING DATA FOR {instrument_code}, skipping ***")
            continue

        # Use the first available trading day
        check_trading_hours(trading_hours, instrument_code)
        all_trading_hours[instrument_code] = trading_hours

    p.close()

    return all_trading_hours