Oracle issues

[andr11111]

Below are oracle failure scenarios we saw during testing:

Scenario 1: skipped days

Description: Oracle fails to run on a specific day (e.g. due to not enough funds or API failures) and a full day is skipped

Impact: Because the MarketContractProxy.getExpiringMarketContract function uses total number of deployed contracts to find the expiring contract to settle, it will be settling contracts with the delay equal to the number of skipped days. E.g. if 2 days is skipped then every day it would settle contract that expired 2 days ago.

Operational Solution: Before restarting the oracle manually deploy contracts for skipped days using deployContract function. That will correct the length of the marketContracts array. If Oracle has already run, similar solution, although less desirable - deploy multiple contracts for today (number of additional contracts equals to number of skipped days).

Guardrails: Oracle address funds monitoring, Oracle error notifications, better/redundant API

Scenario 2: multiple runs during same day

Description: Oracle runs multiple times on the same day creating more contracts than necessary.

Impact: If nothing is done, it looks like this would cause the dailySettlement to try to settle the contract that's about to expire in the future. E.g. if number of incorrectly deployed contracts is 2 then Oracle will try to settle contract expiring 2 days from now. This will lead to no settlement being triggered on actually expiring contracts, which means no user will be able to settle their contracts.

Operational solution: To restore normal working of the protocol it should be paused for the number of days equivalent to the number of incorrectly deployed contracts. Alternatively, oracle script needs to be changed to do settlement and deployment in two separate call - the atomic dailySettlement function will not work correctly

Guardrails: Oracle script should be modified to gracefully exit if same-day contract has already been deployed

[chrismaree]

Possible contract changes to deal with these scenarios:

1. modify the marketContracts array at an arbitrary position.
2. delete arbitrary position in marketContracts and shrink length.
3. guard rails around multiple deployments.