Discover app compilation on node 14

[Marudny]

As mentioned here: dap-ps/infra-dapps#22 Elastic Beanstalk is only in V2 version.
According to the https://docs.aws.amazon.com/elasticbeanstalk/latest/platforms/platforms-retiring.html#platforms-retiring.nodejs the lowest supported version of nodejs is 14.0.0 hence is required to compile discover in nodejs 14 env.

[Marudny]

It's possible to create nodejs virtual env similar to python one.

mkdir -p ~/venvs; cd ~/venvs
nodeenv --node=12.22.11 node-12.22.11
cd node-12.22.11
source bin/activate

I've created nodejs 12.22.11 env to check how compiling process looks like.

[Marudny]

Ok, as expected it doesn't compile successfully:

compiling solidity contracts...
/home/archie/work/dap-ps/discover/.embark/contracts/token/MiniMeToken.sol:273:75: Warning: Unused function parameter. Remove or comment out the variable name to silence this warning.
    function approve(address _spender, uint256 _amount) external returns (bool success) {
                                                                          ^----------^

/home/archie/work/dap-ps/discover/.embark/contracts/token/MiniMeToken.sol:572:5: Warning: Function state mutability can be restricted to view
    function isContract(address _addr) internal returns(bool) {
    ^ (Relevant source part starts here and spans across multiple lines).

assuming Controlled to be an interface
assuming ApproveAndCallFallBack to be an interface
assuming ERC20Token to be an interface
assuming MiniMeTokenInterface to be an interface
assuming TokenController to be an interface
assuming TokenFactory to be an interface
To get more details on interface Smart contracts, go here: https://embark.status.im/docs/troubleshooting.html#Assuming-Contract-to-be-an-interface
deploying contracts
[Discover]: Returned error: Returned error: insufficient funds for gas * price + value
[DiscoverKyberSwap]: Returned error: Returned error: insufficient funds for gas * price + value
Error deploying contracts. Please fix errors to continue.
Finished deploying
[ -d full-build ] || mkdir -p full-build
cp .npmrc full-build/
if [ -f ./full-build/yarn.lock ]; then \
	cmp -s ./back-end/yarn.lock ./full-build/yarn.lock; \
	RETVAL=$?; \
	if [ ! $RETVAL -eq 0 ]; then \
		echo "yarn.lock is different. Removing node_modules and replacingyarn.lock"; \
		rm -rf full-build/yarn.lock full-build/node_modules; \
	fi \
fi
rsync -r --exclude node_modules ./back-end/* ./full-build/
./node_modules/.bin/react-scripts build
Creating an optimized production build...
Browserslist: caniuse-lite is outdated. Please run the following command: `yarn upgrade`
Failed to compile.

Let see how it works on node 14.

gyp verb `which` failed Error: not found: python2
gyp verb `which` failed     at getNotFoundError (/home/archie/work/dap-ps/discover/node_modules/which/which.js:13:12)
gyp verb `which` failed     at F (/home/archie/work/dap-ps/discover/node_modules/which/which.js:68:19)
gyp verb `which` failed     at E (/home/archie/work/dap-ps/discover/node_modules/which/which.js:80:29)
gyp verb `which` failed     at /home/archie/work/dap-ps/discover/node_modules/which/which.js:89:16
gyp verb `which` failed     at /home/archie/work/dap-ps/discover/node_modules/isexe/index.js:42:5
gyp verb `which` failed     at /home/archie/work/dap-ps/discover/node_modules/isexe/mode.js:8:5
gyp verb `which` failed     at FSReqCallback.oncomplete (fs.js:191:21)
gyp verb `which` failed  python2 Error: not found: python2
gyp verb `which` failed     at getNotFoundError (/home/archie/work/dap-ps/discover/node_modules/which/which.js:13:12)
gyp verb `which` failed     at F (/home/archie/work/dap-ps/discover/node_modules/which/which.js:68:19)
gyp verb `which` failed     at E (/home/archie/work/dap-ps/discover/node_modules/which/which.js:80:29)
gyp verb `which` failed     at /home/archie/work/dap-ps/discover/node_modules/which/which.js:89:16
gyp verb `which` failed     at /home/archie/work/dap-ps/discover/node_modules/isexe/index.js:42:5
gyp verb `which` failed     at /home/archie/work/dap-ps/discover/node_modules/isexe/mode.js:8:5
gyp verb `which` failed     at FSReqCallback.oncomplete (fs.js:191:21) {
gyp verb `which` failed   code: 'ENOENT'
gyp verb `which` failed }
gyp verb check python checking for Python executable "python" in the PATH
gyp verb `which` succeeded python /usr/bin/python
gyp ERR! configure error 
gyp ERR! stack Error: Command failed: /usr/bin/python -c import sys; print "%s.%s.%s" % sys.version_info[:3];
gyp ERR! stack   File "<string>", line 1
gyp ERR! stack     import sys; print "%s.%s.%s" % sys.version_info[:3];
gyp ERR! stack                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
gyp ERR! stack SyntaxError: Missing parentheses in call to 'print'. Did you mean print(...)?

Well, it's so old that python2 is required... let's create venv with python2 and try again.

    at /home/archie/work/dap-ps/discover/node_modules/ganache-cli/build/ganache-core.node.cli.js:17:276
    at f.<anonymous> (/home/archie/work/dap-ps/discover/node_modules/ganache-cli/build/ganache-core.node.cli.js:17:2238)
    at f.emit (events.js:400:28)
    at f.destroy (/home/archie/work/dap-ps/discover/node_modules/ganache-cli/build/ganache-core.node.cli.js:29:691909)
    at finish (internal/streams/writable.js:670:14)
    at processTicksAndRejections (internal/process/task_queues.js:82:21)
make: *** [Makefile:53: compile-contracts] Error 1

Got progress.

[Marudny]

ConsenSys-archive/ganache#568

It's a bug in a ganache-cli, installing newer, beta version should help: npm install ganache-core@beta -E
After installing newer version it's required to install required dependencies by calling npm install

[Marudny]

Faced the same error on Node 14 env.

/home/archie/work/dap-ps/discover/.embark/contracts/token/MiniMeToken.sol:273:75: Warning: Unused function parameter. Remove or comment out the variable name to silence this warning.
    function approve(address _spender, uint256 _amount) external returns (bool success) {
                                                                          ^----------^

/home/archie/work/dap-ps/discover/.embark/contracts/token/MiniMeToken.sol:572:5: Warning: Function state mutability can be restricted to view
    function isContract(address _addr) internal returns(bool) {
    ^ (Relevant source part starts here and spans across multiple lines).

assuming Controlled to be an interface
assuming ApproveAndCallFallBack to be an interface
assuming ERC20Token to be an interface
assuming MiniMeTokenInterface to be an interface
assuming TokenController to be an interface
assuming TokenFactory to be an interface
To get more details on interface Smart contracts, go here: https://framework.embarklabs.io/docs/troubleshooting.html#Assuming-Contract-to-be-an-interface
deploying contracts
[Discover]: Returned error: Returned error: insufficient funds for gas * price + value
[DiscoverKyberSwap]: Returned error: Returned error: insufficient funds for gas * price + value
Error deploying contracts. Please fix errors to continue.

[jakubgs]

This is not an error we care about:

Error deploying contracts. Please fix errors to continue.

We just want to compile the solidity contract, we don't care about deploying it. We just need the ABI to be able to build the app.

[jakubgs]

You can get the logs for failed deployment from here:

Here's the failure in /var/log/nodejs/nodejs.log:

Server started on port: 8081
(node:12898) DeprecationWarning: current Server Discovery and Monitoring engine is deprecated, and will be removed in a future version. To use the new Server Discover and Monitoring engine, pass option { useUnifiedTopology: true } to the MongoClient constructor.
Warning: no saslprep library specified. Passwords will not be sanitized
(node:12898) DeprecationWarning: collection.ensureIndex is deprecated. Use createIndexes instead.
Warning: no saslprep library specified. Passwords will not be sanitized
SyntaxError: Unexpected token < in JSON at position 0
    at JSON.parse (<anonymous>)
    at createStrictSyntaxError (/var/app/current/node_modules/body-parser/lib/types/json.js:158:10)
    at parse (/var/app/current/node_modules/body-parser/lib/types/json.js:83:15)
    at /var/app/current/node_modules/body-parser/lib/read.js:121:18
    at invokeCallback (/var/app/current/node_modules/raw-body/index.js:224:16)
    at done (/var/app/current/node_modules/raw-body/index.js:213:7)
    at IncomingMessage.onEnd (/var/app/current/node_modules/raw-body/index.js:273:7)
    at IncomingMessage.emit (events.js:314:20)
    at IncomingMessage.EventEmitter.emit (domain.js:483:12)
    at endReadableNT (_stream_readable.js:1241:12)
    at processTicksAndRejections (internal/process/task_queues.js:84:21)

[jakubgs]

In order to get things building locally I used a Nix shell like this:

NIXPKGS_ALLOW_INSECURE=1 nix-shell

With this shell.nix config:

{ pkgs ? import <nixpkgs> {} }:

pkgs.mkShell {
  name = "discover";
  buildInputs = with pkgs; [
    nodejs-12_x
    (yarn.override { nodejs = nodejs-12_x; })
    python2
  ];
}

I also had to upgrade node-sass or compilations would fail, this also involved purging yarn.lock:

diff --git a/package.json b/package.json
index 9d49596..6ef1eb6 100644
--- a/package.json
+++ b/package.json
@@ -28,7 +28,7 @@
     "history": "^4.7.2",
     "idb": "4.0.3",
     "moment": "^2.24.0",
-    "node-sass": "^4.11.0",
+    "node-sass": "^4.14.1",
     "prop-types": "^15.7.2",
     "rc-slider": "8.6.9",
     "rc-tooltip": "3.7.3",

Newest still supports NodeJS 12: https://github.com/sass/node-sass/releases/tag/v7.0.1

But it fails at build time:

./src/modules/Terms/Terms.module.scss
Error: Node Sass version 7.0.1 is incompatible with ^4.0.0.

[jakubgs]

Actually. After purging all of ~/.yarn and ~/.cache/yarn and node_modules and build in that Nix shell from above without changing package.json it does build successfully:

[nix-shell:~/work/dap-ps/discover]$ yarn run build:dev
yarn run v1.22.18
$ ./node_modules/.bin/cross-env NODE_ENV=development make all
rm -fr full-build 
rm -f app.zip
./node_modules/.bin/embark build "testnet"
...
  adding: logs.log (deflated 77%)
  adding: utils/ (stored 0%)
  adding: utils/authorization-utils.js (deflated 48%)
  adding: .npmrc (deflated 12%)
SUCCESS! Use the app.zip file.
Done in 99.27s.

[nix-shell:~/work/dap-ps/discover]$ ls -l app.zip 
-rw-r--r-- 1 jakubgs jakubgs 10732358 May 24 15:15 app.zip

But the question is, will it work.

[jakubgs]

Started a new deployment:

[jakubgs]

The API works, as far as I can tell:

 > c https://raw.dev.dap.ps/metadata/all | jq -c '.[].details | { name, url, category }'
{"name":"TEH REI","url":"https://teh.rei","category":"COLLECTIBLES"}
{"name":"Matcha","url":"https://matcha.xyz","category":"EXCHANGES"}
{"name":"Gitcoin Grants","url":"https://gitcoin.co/grants","category":"UTILITIES"}
{"name":"1inch Exchange","url":"https://1inch.exchange","category":"EXCHANGES"}
{"name":"1 Inch Exchange","url":"https://1inch.exchange","category":"EXCHANGES"}

Which matches the MongoDB contents:

> db.dappsmetadatas.find({}, {"details.name": 1, "details.url": 1})
{ "_id" : ObjectId("6019bed289c3d16e9199b985"), "details" : { "name" : "TEH REI", "url" : "https://teh.rei" } }
{ "_id" : ObjectId("6045e98b8071110d03266a44"), "details" : { "name" : "Matcha", "url" : "https://matcha.xyz" } }
{ "_id" : ObjectId("6045ea068071110d03266a46"), "details" : { "name" : "Gitcoin Grants", "url" : "https://gitcoin.co/grants" } }
{ "_id" : ObjectId("6045ea4d8071110d03266a48"), "details" : { "name" : "1inch Exchange", "url" : "https://1inch.exchange" } }
{ "_id" : ObjectId("6045eaac8071110d03266a4a"), "details" : { "name" : "1 Inch Exchange", "url" : "https://1inch.exchange" } }

So that's nice.

[jakubgs]

And yet it fails to deploy:

[jakubgs]

I started a rebuild for shits and giggles:

So don't be surprised.

[jakubgs]

I logged into the current dev EB host and found that the DeprecationWarning line are there, but that's it:

[ec2-user@ip-10-0-98-142 ~]$ cat /var/log/nodejs/nodejs.log
Server started on port: 8081
(node:9725) DeprecationWarning: current Server Discovery and Monitoring engine is deprecated, and will be removed in a future version. To use the new Server Discover and Monitoring engine, pass option { useUnifiedTopology: true } to the MongoClient constructor.
Warning: no saslprep library specified. Passwords will not be sanitized
(node:9725) DeprecationWarning: collection.ensureIndex is deprecated. Use createIndexes instead.
Warning: no saslprep library specified. Passwords will not be sanitized