How to delete erroneous instrument from multiple prices

[j1404]

Okay, so I've managed to create nonsensical instrument by making typos and interrupting with Ctrl+C in the wrong place.

Interactive Diagnostic show I have instrument named 'C\x08', which causes update_sampled_contracts to fail. I cannot delete it with Interactive Control, as it is not in the list of valid instruments. So how can I remove this entry from the db?

Your choice? 6
60: Delete instrument from price tables

Your choice? 60
Instrument code?(Return to EXIT)C\x08
C\x08 is not in list ['ALUMINIUM', 'AUD_micro', 'BITCOIN', 'BOBL', 'BRE', 'BRENT-LAST', 'BTP', 'BUND', 'BUXL', 'CAC', 'CAD', 'CAD10', 'CAD5', 'CHF', 'COCOA', 'COFFEE', 'COPPER', 'COPPER-micro', 'CORN', 'CRUDE_W_micro', 'DAX', 'DJSTX-SMALL', 'DOW', 'DX', 'ETHEREUM', 'EU-AUTO', 'EU-BANKS', 'EU-BASIC', 'EU-DIV30', 'EU-HEALTH', 'EU-OIL', 'EU-REALESTATE', 'EU-TECH', 'EU-TRAVEL', 'EU-UTILS', 'EUA', 'EURCHF', 'EURO600', 'EUROSTX', 'EUR_micro', 'FEEDCOW', 'FTSECHINAA', 'FTSECHINAH', 'FTSETAIWAN', 'GAS-LAST', 'GAS_US_mini', 'GBP', 'GOLD_micro', 'IBEX_mini', 'IRON', 'JPY', 'KOSDAQ', 'KOSPI_mini', 'KR10', 'KRWUSD_mini', 'LEANHOG', 'LIVECOW', 'MILLWHEAT', 'MSCISING', 'MUMMY', 'MXP', 'NASDAQ_micro', 'NIKKEI400', 'NZD', 'OAT', 'OATIES', 'OJ', 'PALLAD', 'PLAT', 'REDWHEAT', 'RUBBER', 'RUSSELL', 'SEK', 'SHATZ', 'SILVER', 'SMI', 'SMI-MID', 'SOYBEAN', 'SOYBEAN_mini', 'SOYOIL', 'SP400', 'SP500_micro', 'SPI200', 'TOPIX', 'US-DISCRETE', 'US-ENERGY', 'US-FINANCE', 'US-HEALTH', 'US-INDUSTRY', 'US-MATERIAL', 'US-REALESTATE', 'US-STAPLES', 'US-TECH', 'US-UTILS', 'US10', 'US10U', 'US2', 'US20', 'US30', 'US5', 'V2X', 'VIX', 'WHEAT', 'YENEUR', 'ZAR'] derived from source: single

[tgibson11]

This doesn't solve your immediate problem, but one way this could be addressed in the code...

Wherever existing data is used to produce a list of instruments, that list should be the intersection of the data and the instrument config.

So any garbage instruments that exist in the data would just be ignored if they aren't also in the config.

There might be a few exceptions: diagnostics, deleting. Although that could easily be worked around by just adding a temporary config record.

[tgibson11]

Then again, maybe the existing ignore_instruments config would do the same thing

[frankcodespoorly]

I think the quickest fix would be to drop the current mongo database with the issue and restore a version prior to the error, then update prices from there.

[j1404]

Tried to add the instrument with and without backslash into all configs I could find, but no luck. Excluding in private_config doesn't help here. Is there a way to drop single instrument from mongo? Or perhaps drop the futures_multiple_prices collections?

I'm still experimenting with the system so not a big problem if everything breaks. Would be useful to know how to handle these situations in production.

[j1404]

I found it and got rid of it, here's how:

use arctic_production

db.futures_multiple_prices.version_nums.find({'symbol': 'C\x08'})
{ "_id" : ObjectId("656621aab4ad74337ae33b42"), "symbol" : "C\b", "version" : 1 }

db.futures_multiple_prices.versions.find({'symbol': 'C\x08'})
{ "_id" : ObjectId("656621aa9e76de2b8220e304"), "arctic_version" : 1079002, "symbol" : "C\b", "version" : 1, "metadata" : null, "dtype" : "[('index', '<f8'), ('CARRY', '<f8'), ('CARRY_CONTRACT', '<f8'), ('FORWARD', '<f8'), ('FORWARD_CONTRACT', '<f8'), ('PRICE', '<f8'), ('PRICE_CONTRACT', '<f8')]", "shape" : [ -1 ], "dtype_metadata" : { "index" : [ "index" ], "columns" : [ "CARRY", "CARRY_CONTRACT", "FORWARD", "FORWARD_CONTRACT", "PRICE", "PRICE_CONTRACT" ] }, "type" : "pandasdf", "up_to" : 0, "sha" : BinData(0,"2jmj7l5rSw0yVb/vlWAYkK/YBwk="), "FW_POINTERS_CONFIG" : "DISABLED", "base_sha" : BinData(0,"2jmj7l5rSw0yVb/vlWAYkK/YBwk="), "segment_count" : 0, "append_size" : 0, "append_count" : 0 }

db.futures_multiple_prices.version_nums.remove({'symbol': 'C\x08'})
WriteResult({ "nRemoved" : 1 })

db.futures_multiple_prices.versions.remove({'symbol': 'C\x08'})
WriteResult({ "nRemoved" : 1 })