Merge develop into futures-implementation (#1326)

* Introducing new integration tests in CI (#1287)

* Introducing integration tests in CI

* Improved integration tests in CI and removed redundant prod tests

* Bugfix on integration test task

* More verbose on deployer error

* Refactoring of Exchanger.sol to reduce size on OVM (#1291)

* Minor fix for integration tests (#1295)

* Clean state on dual integration tests plus slightly better exchange tests

* Minor fix to integration tests

* Disable some exchanging integration tests for now

* Extra prod tests (#1299)

* Add issuance prod tests

* Add erc20 behavior

* Tweaks on incoming integration tests

* Minor fix to integrationt ests

Co-authored-by: Alejandro Santander <Palebluedot@gmail.com>

* Audit feedback and spring cleaning (#1300)

* Port more integration tests (#1288)

* Basic integration tests for L2 deposits

* Basic integration tests for L2 deposits

* Polish deposit integration tests

* Progress porting withdrawal integration tests

* Working withdrawals in new integration tests

* Using watcher tool more

* Unify action and actionTo in integration tests

* Progress porting integration tests

* Remove comment

* Implemented exchanges in new integration tests

* Bugfix in integrationt est task

* Basic forking in integration tests

* Include fork tests in CI

* Minor fixes for prod tests

* port migrateEscrow test to integration dual

* Fix CI

* Fix CI

Co-authored-by: Leonardo Massazza <lmassazza+github@gmail.com>
Co-authored-by: Yannis <i.stamelakos@gmail.com>

* Better way to get SNX in integration tests (#1303)

* Better way to get SNX

* 2192

* Introduces forking via integration tests (#1307)

* Introduces forking with integration tests

* Compile and deploy on fork tests

* Fix old ovm prod tests

* Replace web3 with ethers in deployment (#1271)

* add wrapper object to hold web3 and ethers

* Replace web3-utils with ethers

* - replace web3.utils with ethers counterpart
- create account with ethers instead of web3

* corrections to failed tests

* propagate Deploy changes fix (test:publish green)

* document the provider addition to Deployer

* document the provider addition to Deployer

* fix require

* fix require

* Clean install

* Update to develop

Co-authored-by: Alejandro Santander <Palebluedot@gmail.com>

* Updates optimism dependencies (#1308)

* Updated optimism deps

* Update usage of dep in code

* Add hardhat-ethers dep required by smock

* Add await in unit test

* Removed @gas-skip (#1309)

* Updated optimism deps

* Update usage of dep in code

* Add hardhat-ethers dep required by smock

* Removed @gas-skip

* Add await in unit test

* Disables prod tests from CI (#1311)

* Reordering the deploy script for sanity (#1304)

* Add integration tests for settle and claim (#1310)

* Support settlements in exchanging behavior

* Testing claims in integration tests

* Address PR feedback, increase timeouts, improve test for forking

* Bigger tolerance for debt comparison

* Approve bridge tokens during bootstrap

* Add SynthsUSD integration (L1 and L2) tests (#1312)

* Add support to SynthsUSD in L1 and L2 integration tests

* Add support to SynthsUSD in L1 and L2 integration tests

* style fixes

Co-authored-by: Alejandro Santander <Palebluedot@gmail.com>

* Port migrateEscrow and depositAndMigrateEscrow tests  (#1306)

* ported. Some tests still failing

* wip

* migrateEscrow test ported

* accept multiples hashes in watchers

* Reduce migrateEscrow dual test to dual scope

* ported depositAndMigrateEscrow test

* update test wording

Co-authored-by: Alejandro Santander <Palebluedot@gmail.com>

* Migration for bulk staking reward upgrade (#1301)

* Deleted everything related to prod tests (#1315)

* Adds ether wrapper integration tests (#1319)

* Basic integration tests for ether wrapper

* Fix timeout in integration tests

* Abstract eth wrapper behavior in integration tests

* Support ether wrapper integration tests in mainnet forks

* Pin ops to a particular commit that is known to work in CI

* Processed Leo's feeedback on PR

* Enables separate folder compilation for integration tests (#1322)

* Enables separate folder compilation for integration tests

* Clean install

* add futures contracts to local-ovm config.json

* fix: only deploy futures on OVM

* deploy EmptyFuturesMarketManger on L1

Since we modify a couple L1 contracts, such as the DebtCache, to incorporate futures debt, we need to deploy a FuturesMarketManager on L1. This uses our empty contract pattern we have in other places, eg EmptyEtherCollateral.

* Check that events are properly emitted when snapshotting excluded debt.

* Remove unnecessary mock in Synth test.

* add simulated rates for futures market assets

* remove check for invalid rates, as it isn't valid for L2

This check calls Issuer.availableCurrencyKeys, which will miss some assets that
are required to have exchange rates but aren't technically enabled for issuing yet
on L2.

* add missing upgrade to SystemSettings

* Revert "remove check for invalid rates, as it isn't valid for L2"

This reverts commit c382f2c.

* fix check for invalid rates on L2

* fix: remove redundant SIP's

Co-authored-by: Alejandro Santander <Palebluedot@gmail.com>
Co-authored-by: justin j. moses <justin@synthetix.io>
Co-authored-by: Yannis <i.stamelakos@gmail.com>
Co-authored-by: Leonardo Massazza <lmassazza+github@gmail.com>
Co-authored-by: Anton Jurisevic <zyzek@users.noreply.github.com>

Author: 6 people

.circleci/config.yml

@@ -26,19 +26,7 @@ jobs:
       - run: npx hardhat compile --optimizer --fail-oversize
       - run: rm -rf build # force a clean build
       - run: npx hardhat compile --use-ovm --optimizer --fail-oversize
-  job-lint:
-    working_directory: ~/repo
-    docker:
-      - image: synthetixio/docker-node:14.16-focal
-        auth:
-          username: $DOCKERHUB_USERNAME
-          password: $DOCKERHUB_TOKEN
-    steps:
-      - checkout
-      - attach_workspace:
-          at: .
-      - run: npm run lint
-  job-pack-browser:
+  job-fork-tests:
     working_directory: ~/repo
     docker:
       - image: synthetixio/docker-node:14.16-focal
@@ -49,94 +37,13 @@ jobs:
       - checkout
       - attach_workspace:
           at: .
-      - run: npm run pack
-      - store_artifacts:
-          path: browser.js
-  job-prepare:
-    working_directory: ~/repo
-    docker:
-      - image: synthetixio/docker-node:14.16-focal
-        auth:
-          username: $DOCKERHUB_USERNAME
-          password: $DOCKERHUB_TOKEN
-    steps:
-      - checkout
-      - restore_cache:
-          keys:
-            - v4-dependencies-{{ checksum "package-lock.json" }}
-      - run: npm install
-      - save_cache:
-          key: v4-dependencies-{{ checksum "package-lock.json" }}
-          paths:
-            - node_modules
-      - persist_to_workspace:
-          root: .
-          paths:
-            - node_modules
-  job-prod-diff-tests-local-ovm:
-    working_directory: ~/repo
-    docker:
-      - image: synthetixio/docker-node:14.16-focal
-        auth:
-          username: $DOCKERHUB_USERNAME
-          password: $DOCKERHUB_TOKEN
-    steps:
-      - checkout
-      - attach_workspace:
-          at: .
-      - run: node publish build --test-helpers
-      - run: npx hardhat compile --force
       - run:
-          command: npx hardhat node
+          command: npm run fork
           background: true
       - cmd-wait-for-port:
           port: 8545
-      - run: node publish deploy --network local --fresh-deploy --yes --use-ovm --ignore-safety-checks --ignore-custom-parameters --deployment-path ./publish/deployed/local-ovm
-      - run: npm run test:prod -- --no-compile --use-ovm --patch-fresh-deployment --deployment-path ./publish/deployed/local-ovm
-  job-prod-diff-tests-local:
-    working_directory: ~/repo
-    docker:
-      - image: synthetixio/docker-node:14.16-focal
-        auth:
-          username: $DOCKERHUB_USERNAME
-          password: $DOCKERHUB_TOKEN
-    steps:
-      - checkout
-      - attach_workspace:
-          at: .
-      - run: node publish build --test-helpers
-      - run:
-          command: npx hardhat node
-          background: true
-      - cmd-wait-for-port:
-          port: 8545
-      - run: node publish deploy --network local --fresh-deploy --yes
-      - run: npm run test:prod -- --patch-fresh-deployment
-  job-prod-diff-tests:
-    working_directory: ~/repo
-    docker:
-      - image: synthetixio/docker-node:14.16-focal
-        auth:
-          username: $DOCKERHUB_USERNAME
-          password: $DOCKERHUB_TOKEN
-    resource_class: large
-    steps:
-      - checkout
-      - attach_workspace:
-          at: .
-      - run: node publish build
-      - run:
-          command: npx hardhat node --target-network mainnet
-          background: true
-      - cmd-wait-for-port:
-          port: 8545
-      - run: node publish prepare-deploy --network mainnet
-      - run: node publish deploy --ignore-safety-checks --add-new-synths --use-fork --yes --network mainnet
-      - run: npm run test:prod:gas -- --target-network mainnet --patch-fresh-deployment
-      - run: npx codechecks codechecks.prod.yml
-      - store_artifacts:
-          path: test-gas-used-prod.log
-  job-prod-tests-ovm:
+      - run: npx hardhat test:integration:l1 --compile --deploy --fork
+  job-integration-tests:
     working_directory: ~/repo
     machine:
       image: ubuntu-2004:202010-01
@@ -152,8 +59,9 @@ jobs:
             git clone git@github.com:ethereum-optimism/optimism.git
             cd optimism
             git fetch
-            git checkout develop
-            git pull origin develop
+            git checkout master
+            git pull origin master
+            git checkout 86708bb5758cd2b647b3ca2be698beb5aa3af81f
             yarn
             yarn build
             cd ops
@@ -171,14 +79,18 @@ jobs:
       - cmd-wait-for-port:
           port: 9545
       - run:
-          name: Deploy OVM Synthetix instances
+          name: Run isolated layer 1 integration tests
+          command: |
+            npx hardhat test:integration:l1 --compile --deploy --provider-port 9545
+      - run:
+          name: Run isolated layer 2 integration tests
           command: |
-            node publish deploy-ovm-pair
+            npx hardhat test:integration:l2 --compile --deploy
       - run:
-          name: Run OVM production tests
+          name: Run dual layer 1 and layer 2 integration tests
           command: |
-            npm run test:prod:ovm
-  job-prod-tests:
+            npx hardhat test:integration:dual --deploy-evm --deploy-ovm --connect
+  job-lint:
     working_directory: ~/repo
     docker:
       - image: synthetixio/docker-node:14.16-focal
@@ -189,15 +101,42 @@ jobs:
       - checkout
       - attach_workspace:
           at: .
-      - run:
-          command: npx hardhat node --target-network mainnet
-          background: true
-      - cmd-wait-for-port:
-          port: 8545
-      - run: npm run test:prod:gas -- --target-network mainnet
-      - run: npx codechecks codechecks.prod.yml
+      - run: npm run lint
+  job-pack-browser:
+    working_directory: ~/repo
+    docker:
+      - image: synthetixio/docker-node:14.16-focal
+        auth:
+          username: $DOCKERHUB_USERNAME
+          password: $DOCKERHUB_TOKEN
+    steps:
+      - checkout
+      - attach_workspace:
+          at: .
+      - run: npm run pack
       - store_artifacts:
-          path: test-gas-used-prod.log
+          path: browser.js
+  job-prepare:
+    working_directory: ~/repo
+    docker:
+      - image: synthetixio/docker-node:14.16-focal
+        auth:
+          username: $DOCKERHUB_USERNAME
+          password: $DOCKERHUB_TOKEN
+    steps:
+      - checkout
+      - restore_cache:
+          keys:
+            - v4-dependencies-{{ checksum "package-lock.json" }}
+      - run: npm install
+      - save_cache:
+          key: v4-dependencies-{{ checksum "package-lock.json" }}
+          paths:
+            - node_modules
+      - persist_to_workspace:
+          root: .
+          paths:
+            - node_modules
   job-static-analysis:
     working_directory: ~/repo
     docker:
@@ -384,23 +323,10 @@ workflows:
       - job-test-deploy-script:
           requires:
             - job-prepare
-      - job-prod-tests:
-          requires:
-            - job-prepare
-      - job-prod-tests-ovm:
-          name: job-prod-tests-ovm
-          requires:
-            - job-prepare
-      - job-prod-diff-tests-local:
-          name: job-prod-diff-tests-local
-          requires:
-            - job-prepare
-      - job-prod-diff-tests-local-ovm:
-          name: job-prod-diff-tests-local-ovm
+      - job-fork-tests:
           requires:
             - job-prepare
-      - job-prod-diff-tests:
-          name: job-prod-diff-tests-mainnet
+      - job-integration-tests:
           requires:
             - job-prepare
       - job-pack-browser:

.circleci/src/jobs/job-fork-tests.yml

@@ -0,0 +1,12 @@
+# Starts a fork of mainnet, deploys the latest release, and runs L1 integration tests
+{{> job-header.yml}}
+steps:
+  - checkout
+  - attach_workspace:
+      at: .
+  - run:
+      command: npm run fork
+      background: true
+  - cmd-wait-for-port:
+        port: 8545
+  - run: npx hardhat test:integration:l1 --compile --deploy --fork

.circleci/src/jobs/job-prod-tests-ovm.yml renamed to .circleci/src/jobs/job-integration-tests.yml

@@ -11,8 +11,9 @@ steps:
         git clone git@github.com:ethereum-optimism/optimism.git
         cd optimism
         git fetch
-        git checkout develop
-        git pull origin develop
+        git checkout master
+        git pull origin master
+        git checkout 86708bb5758cd2b647b3ca2be698beb5aa3af81f
         yarn
         yarn build
         cd ops
@@ -30,10 +31,14 @@ steps:
   - cmd-wait-for-port:
         port: 9545
   - run:
-      name: Deploy OVM Synthetix instances
+      name: Run isolated layer 1 integration tests
       command: |
-        node publish deploy-ovm-pair
+        npx hardhat test:integration:l1 --compile --deploy --provider-port 9545
   - run:
-      name: Run OVM production tests
+      name: Run isolated layer 2 integration tests
       command: |
-        npm run test:prod:ovm
+        npx hardhat test:integration:l2 --compile --deploy
+  - run:
+      name: Run dual layer 1 and layer 2 integration tests
+      command: |
+        npx hardhat test:integration:dual --deploy-evm --deploy-ovm --connect

.circleci/src/jobs/job-prod-diff-tests-local-ovm.yml

@@ -25,21 +25,15 @@ jobs:
       {{> require-prepare.yml}}
 
   # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-  # Prod tests
+  # Fork tests
   # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-  - job-prod-tests:
+  - job-fork-tests:
       {{> require-prepare.yml}}
-  - job-prod-tests-ovm:
-      name: job-prod-tests-ovm
-      {{> require-prepare.yml}}
-  - job-prod-diff-tests-local:
-      name: job-prod-diff-tests-local
-      {{> require-prepare.yml}}
-  - job-prod-diff-tests-local-ovm:
-      name: job-prod-diff-tests-local-ovm
-      {{> require-prepare.yml}}
-  - job-prod-diff-tests:
-      name: job-prod-diff-tests-mainnet
+
+  # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+  # Integration tests
+  # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+  - job-integration-tests:
       {{> require-prepare.yml}}
 
   # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~