feat(mojaloop/#3078): get /trans callback is wrong for exp trans (moj…

…aloop#938) feat(mojaloop/#3078): get /transfers callback is wrong for expired transfers - mojaloop/project#3078 - removed incorrect logic from get-handler which would throw an exception when the transfer-state was TransferInternalState.EXPIRED_RESERVED - updated unit tests to reflect this change chore: enhancements for local functional and dev environment setups - updated main readme with instructions on how to run functional tests using the ml-core-test-harness - added docker compose for local dev with updated instructions on main readme - added postman script to use against populateData scripts under test/util folder chore: enhancements for local functional and dev environment setups - updated main readme with instructions on how to run functional tests using the ml-core-test-harness - added docker compose for local dev with updated instructions on main readme chore: updated dependencies and audits with approrpriate commets
oderayi · Jan 27, 2023 · 70bdf18 · 70bdf18
1 parent 5ca3bd9
commit 70bdf18
Show file tree

Hide file tree

Showing 13 changed files with 1,165 additions and 343 deletions.
diff --git a/.ncurc.yaml b/.ncurc.yaml
@@ -5,5 +5,11 @@ reject: [
   # TODO: New versions from 2.2.0 onwards introduce a newer incompatible version of the ILP-Packet that is not compatible with the Mojaloop Specification
   "ilp-packet",
   # TODO: v6+ (ref: https://github.com/sindresorhus/get-port/releases/tag/v6.0.0) is an ESM library and thus not compatible with CommonJS. Future story needed to resolve.
-  "get-port"
+  "get-port",
+  # There seems to be an issue with seeding or migration with integration tests
+  # that lead to some tests failing down the line.
+  # Specifically test case "update transfer state to COMMITTED by FULFIL request"
+  # "savePayeeTransferResponse::failure Cannot read properties of undefined (reading 'settlementWindowId')"
+  # TODO: More investigation
+  "knex"
 ]
diff --git a/README.md b/README.md
@@ -26,6 +26,8 @@ The following documentation represents the services, APIs and endpoints responsi
   - [Logging](#logging)
   - [Tests](#tests)
     - [Running Integration Tests interactively](#running-integration-tests-interactively)
+  - [Development environment](#development-environment)
+  - [Auditing Dependencies](#auditing-dependencies)
   - [Container Scans](#container-scans)
   - [Automated Releases](#automated-releases)
     - [Potential problems](#potential-problems)
@@ -45,7 +47,7 @@ The Central Ledger has many options that can be configured through environment v
 | CLEDG\_DATABASE_URI   | The connection string for the database the central ledger will use. Postgres is currently the only supported database. | postgres://\<username>:\<password>@localhost:5432/central_ledger |
 | CLEDG\_PORT | The port the API server will run on. | 3000 |
 | CLEDG\_ADMIN_PORT | The port the Admin server will run on. | 3001 |
-| CLEDG\_HOSTNAME | The URI that will be used to create and validate links to resources on the central ledger.  | http://central-ledger |
+| CLEDG\_HOSTNAME | The URI that will be used to create and validate links to resources on the central ledger.  | <http://central-ledger> |
 | CLEDG\_ENABLE\_BASIC_AUTH | Flag to enable basic auth protection on endpoints that require authorization. Username and password would be the account name and password. | false |
 | CLEDG\_ENABLE\_TOKEN_AUTH | Flag to enable token protection on endpoints that require authorization. To create a token, reference the [API documentation](API.md). | false |
 | CLEDG\_LEDGER\_ACCOUNT_NAME | Name of the account setup to receive fees owed to the central ledger. If the account doesn't exist, it will be created on start up. | LedgerName |
@@ -119,6 +121,61 @@ If you want to run integration tests in a repetitive manner, you can startup the
     npm run test:int
   ```
 
+### Running Functional Tests
+
+If you want to run functional tests locally utilizing the [ml-core-test-harness](https://github.com/mojaloop/ml-core-test-harness), you can run the following commands:
+
+  ```bash
+  git clone --depth 1 --branch v0.0.2 https://github.com/mojaloop/ml-core-test-harness.git ./IGNORE/ml-core-test-harness
+  ```
+
+  ```bash
+  docker build -t mojaloop/central-ledger:local .
+  ```
+
+  ```bash
+  cd IGNORE/ml-core-test-harness
+  export CENTRAL_LEDGER_VERSION=local
+  docker-compose --project-name ttk-func --ansi never --profile all-services --profile ttk-provisioning --profile ttk-tests up -d
+  ```
+
+  Check test container logs for test results
+
+  Or access TTK UI using the following URI: <http://localhost:9660>
+
+  TTK Test files:
+      - Test Collection: ./IGNORE/ml-core-test-harness/docker/ml-testing-toolkit/test-cases/collections/tests/p2p.json
+      - Env Config: ./IGNORE/ml-core-test-harness/docker/ml-testing-toolkit/test-cases/environments/default-env.json
+
+## Development environment
+
+  Start Docker dependant Services
+
+  ```bash
+  docker compose -f ./docker-compose.yml -f docker-compose.dev.yml up -d
+  ```
+
+  Start local Central-Ledger Service
+
+  ```bash
+  npm start
+  ```
+
+  Populate Test Data
+
+  ```bash
+  sh ./test/util/scripts/populateTestData.sh
+  ```
+
+  View Logs for Mockserver (i.e. Payee Receiver) and ML-API-Adapter:
+
+  ```bash
+  docker logs -f mockserver
+  docker logs -f cl_ml-api-adapter
+  ```
+
+  Postman Test Collection: [./test/util/postman/CL-Local Docker Test.postman_collection.json](./test/util/postman/CL-Local%20Docker%20Test.postman_collection.json)
+
 ## Auditing Dependencies
 
 We use `npm-audit-resolver` along with `npm audit` to check dependencies for node vulnerabilities, and keep track of resolved dependencies with an `audit-resolve.json` file.
@@ -144,8 +201,8 @@ As part of our CI/CD process, we use anchore-cli to scan our built docker contai
 If you find your release builds are failing, refer to the [container scanning](https://github.com/mojaloop/ci-config#container-scanning) in our shared Mojaloop CI config repo. There is a good chance you simply need to update the `mojaloop-policy-generator.js` file and re-run the circleci workflow.
 
 For more information on anchore and anchore-cli, refer to:
-- [Anchore CLI](https://github.com/anchore/anchore-cli)
-- [Circle Orb Registry](https://circleci.com/orbs/registry/orb/anchore/anchore-engine)
+    - [Anchore CLI](https://github.com/anchore/anchore-cli)
+    - [Circle Orb Registry](https://circleci.com/orbs/registry/orb/anchore/anchore-engine)
 
 ## Automated Releases
 

diff --git a/audit-ci.jsonc b/audit-ci.jsonc
@@ -19,6 +19,18 @@
     "GHSA-g64q-3vg8-8f93",
     "GHSA-5854-jvxx-2cg9",
     "GHSA-w5p7-h5w8-2hfq",
-    "GHSA-p9pc-299p-vxgp"
+    "GHSA-p9pc-299p-vxgp",
+    // TODO: Investigate issue with json5 lib as this is due to an underlying dependant lib dependency.
+    "GHSA-9c47-m6qq-7p4h", // https://github.com/advisories/GHSA-9c47-m6qq-7p4h
+    // TODO: Upgrade jsonwebtoken in the central-services-shared lib --> https://github.com/mojaloop/project/issues/3097
+    "GHSA-qwph-4952-7xr6", // https://github.com/advisories/GHSA-qwph-4952-7xr6
+    "GHSA-27h2-hvpr-p74q", // https://github.com/advisories/GHSA-27h2-hvpr-p74q
+    // Unable to upgrade Knex due to the following issue:
+    // # There seems to be an issue with seeding or migration with integration tests
+    // # that lead to some tests failing down the line.
+    // # Specifically test case "update transfer state to COMMITTED by FULFIL request"
+    // # "savePayeeTransferResponse::failure Cannot read properties of undefined (reading 'settlementWindowId')"
+    // TODO: More investigation --> https://github.com/mojaloop/project/issues/3096
+    "GHSA-4jv9-3563-23j3" // https://github.com/advisories/GHSA-4jv9-3563-23j3
   ]
 }
diff --git a/docker-compose.dev.yml b/docker-compose.dev.yml
@@ -0,0 +1,63 @@
+version: "3.7"
+services:
+  ## startup central-ledger to run any migrations. You will need to manually start central-ledger using `npm start`
+  central-ledger:
+    image: mojaloop/central-ledger:local
+    deploy:
+      replicas: 1
+    user: root
+    build:
+      context: .
+      cache_from:
+        - mojaloop/central-ledger
+        - central-ledger
+    container_name: cl_central-ledger
+    command:
+      - "sh"
+      - "-c"
+      - "sh /opt/wait-for/wait-for-central-ledger.sh && npm run migrate"
+    ports:
+      - "3001:3001"
+    volumes:
+       - ./docker/central-ledger/default.json:/opt/app/config/default.json
+       - ./docker/wait-for:/opt/wait-for
+    environment:
+      - LOG_LEVEL=info
+      - CSL_LOG_TRANSPORT=file
+      - CLEDG_MONGODB__DISABLED=false
+    networks:
+      - cl-mojaloop-net
+    depends_on:
+      - mysql
+      - kafka
+      - objstore
+    healthcheck:
+      test: ["CMD", "sh", "-c" ,"apk --no-cache add curl", "&&", "curl", "http://localhost:3001/health"]
+      timeout: 20s
+      retries: 10
+      interval: 30s
+
+  ## override ml-api-adapter to ignore health checks for central-ledger
+  ml-api-adapter:
+    image: mojaloop/ml-api-adapter:latest
+    container_name: cl_ml-api-adapter
+    deploy:
+      replicas: 1
+    user: root
+    command:
+      - "sh"
+      - "-c"
+      - "sh /opt/wait-for/wait-for-ml-api-adapter.sh && node src/api/index.js"
+    ports:
+      - "3000:3000"
+    volumes:
+      - ./docker/ml-api-adapter/default.json:/opt/app/config/default.json
+      - ./docker/wait-for:/opt/wait-for
+    environment:
+      - LOG_LEVEL=info
+    networks:
+      - cl-mojaloop-net
+    depends_on:
+      - kafka
+    ## We override the healthcheck here so we can start ml-api-adapter without the central-ledger running in docker, so we can start it locally
+    healthcheck: {}