Skip to content

Deploying the portal

Jump to bottom
Thomas Schaffter edited this page Mar 24, 2020 · 34 revisions

Overview

This section describes how to build and deploy the portal and its components for development and production environments. Instructions are provided to either deploy the portal manually or using docker-compose.

Portal components

The portal is composed of the following components:

  • Portal
    • MongoDB
    • Provenance
      • Neo4j

Notes:

  • The docker images of the portal and its components (e.g. provenance) are hosted on docker.synapse.org (use docker login docker.synapse.org to login).
  • The docker images of the portal and its components are currently part of a private repository. You must be given Read access to this repository by one of the administrators and have a certified Synapse user account to be able to pull the docker images.

Development environment

This section describes how to run the portal for development purpose.

If an error occurs, a solution may be found in the FAQ.

Cloning repositories

First, start by cloning the following repositories:

Installing the portal dependencies

From the portal directory, run npm install (or npm i).

Generating SSL certificates

The portal is designed to run over HTTPS. In production environment, the communications between the portal and its components are run over SSL (the current exceptions are the portal-provenance and provenance-neo4j communications). It is recommended to use SSL certificates generated by a trusted Certificate Authority (CA). However, it may be convenient to use self-signed certificates for the sake of development.

From the portal directory, run the following commands to generate a self-signed certificate that the portal will use.

Note: Keep the same directory and file names.

export CERTS_DIR=certs 
mkdir $CERTS_DIR
openssl req -new -newkey rsa:4096 -days 365 -nodes -x509 \
  -subj "/C=US/ST=Washington/L=Seattle/O=Sage Bionetworks/CN=www.sagebionetworks.org" \
  -keyout $CERTS_DIR/server.key -out $CERTS_DIR/server.cert

We are using this opportunity to generate a certificate in PEM format to run MongoDB over SSL, which will be used in production environment. This can be achieved by concatenating server.key and server.cert (assumes that the above code as been run):

cat $CERTS_DIR/server.key $CERTS_DIR/server.cert > $CERTS_DIR/mongodb.pem

Setting the environment variables

The portal and its components are configured using environment variables.

The file envvars-dev.sample provides an example of configuration that can be used to run the portal.

  1. Copy the file envvars-dev.sample to envvars-dev
  2. Review the content of envvars-dev
  3. source envvars-dev

See the section Configuring the portal for detailed information on how to configure different parts of the portal using its environment variables.

Starting MongoDB

MongoDB is used by the portal to store data. The following command deploys the official docker image mongo. Here we assume that the environment variable MONGO_PORT is set (see Section Setting the environment variables).

docker run -p ${MONGO_PORT}:${MONGO_PORT} --name mongo -d mongo

To test that MongoDB is running properly, let's connect to the instance using the MongoDB client:

$ mongo
MongoDB shell version v4.0.10
connecting to: mongodb://127.0.0.1:27017/?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("afd9f563-7873-4805-96b3-75c0550551bd") }
MongoDB server version: 4.0.10
Server has startup warnings: 
2019-07-27T21:35:31.504+0000 I STORAGE  [initandlisten] 
2019-07-27T21:35:31.504+0000 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2019-07-27T21:35:31.504+0000 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
2019-07-27T21:35:32.332+0000 I CONTROL  [initandlisten] 
2019-07-27T21:35:32.332+0000 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2019-07-27T21:35:32.332+0000 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2019-07-27T21:35:32.332+0000 I CONTROL  [initandlisten] 
---
Enable MongoDB's free cloud-based monitoring service, which will then receive and display
metrics about your deployment (disk utilization, CPU, operation statistics, etc).

The monitoring data will be available on a MongoDB website with a unique URL accessible to you
and anyone you share the URL with. MongoDB may use this information to make product
improvements and to suggest MongoDB products and deployment options to you.

To enable free monitoring, run the following command: db.enableFreeMonitoring()
To permanently disable this reminder, run the following command: db.disableFreeMonitoring()
---

> exit
bye

Starting Provenance

This section describe how to run the latest release on the develop branch of https://github.com/Sage-Bionetworks/prov-service. Here we assume that the environment variables NEO4J_USERNAME and NEO4J_PASSWORD are set (see Section https://github.com/Sage-Bionetworks/PHCCollaborationPortal/wiki/Deploying-the-portal#setting-the-environment-variables). Then, start the provenance service and the Neo4j database it depends on with:

$ git pull
$ docker-compose -f docker-compose.sbcp.yml up
...
prov-server    | INFO:werkzeug: * Running on http://0.0.0.0:8080/ (Press CTRL+C to quit)

or docker-compose up -d to detach and thus keep the containers running in the background.

Starting the portal

In development environment, there are two processes that need to be started:

Important: When using two terminal sessions to run the following commands, remember to source the environment variables for each session beforehand.

  1. source envvars-dev && sudo -E npm run start:server
  2. source envvars-dev && sudo -E npm run start:client

The first command runs the Express server of the portal in development environment. If one of the file that is used by the backend is modified (typically in the directory server), the backend will automatically restart. Note that if the server is configured to seed the DB, each restart of the backend will reset the content of the database (unless specifying APP_INIT_DB_SEED_NAME= without any value).

In a similar fashion, saving a modification to one of the front-end file (typically in the directory client), the dev server ran using the second command will automatically reload.

Accessing the portal

The portal should now be available at the address https://localhost (assumes that DOMAIN=https://localhost in envvars-dev).

Note: Open the browser console to see the log generated by the client side of the portal.

Running unit tests

It is strongly recommended to run the unit tests before committing changes to a GitHub repository. The following command runs both server and client side tests (ideally run the test from a terminal session that do not have portal environment variables set).

$ npm run test
...
27 07 2019 22:04:51.933:INFO [launcher]: Starting browser ChromeHeadless
27 07 2019 22:04:53.384:INFO [HeadlessChrome 77.0.3835 (Linux 0.0.0)]: Connected on socket C0DqMDI8IfO7qMxRAAAA with id 12291673

  Util
    ✓ Has a safeCb function

HeadlessChrome 77.0.3835 (Linux 0.0.0): Executed 1 of 1 SUCCESS (0.008 secs / 0.001 secs)
TOTAL: 1 SUCCESS

[22:04:54] Finished 'test:client' after 30 s
[22:04:54] Finished 'test' after 2.1 min

Production environment

This section describes how to build and run the portal manually in production environment. Instructions are also given to deploy the portal and all its components using docker-compose.

Building the portal manually for production environment

This section describes how to build the portal for production environment. First, open a new terminal session (to prevent inheriting environment variables used in development environment) and run the following command from the portal directory.

npm run build

Upon successful completion, the portal build is now available in the folder dist/.

Run the portal manually

First, copy the file envvars-prod.sample to envvars-prod. Note that this file contains a configuration that requires an instance of MongoDB configured to run over SSL and that requires authentication (see docker-compose.yml). As we currently have an instance of MongoDB running without SSL or authentication, the following changes must be made to envvars-prod.

To disable user authentication to the MongoDB instance:

  • MONGODB_USER= (empty value)
  • MONGODB_PASSWORD= (empty value)

To disable connecting to the MongoDB instance over SSL:

  • MONGODB_SSL=false

The following command should then starts the portal in production environment:

source envvars-prod && sudo -E node dist/server

Deploying the portal stack using docker-compose

This section describes how to deploy the portal and all its components as docker containers.

Important: All the credentials included in the environment variable sample must be replaced by complex credentials randomly generated before deploying the portal for actual users.

after building the portal docker image

Here we describe how to build the portal docker image as part of the docker-compose deployment. The alternative is to deploy the portal stack using the official docker image of the portal (see the next section).

  1. Copy envvars-prod.sample to envvars-prod
  2. Review the content of envvars-prod
  3. source envvars-prod && docker-compose -f docker-compose.yml up --build

The portal should now be accessible on https://localhost (assuming the environment variable DOMAIN=https://localhost).

using the official portal docker image

  1. Copy envvars-prod.sample to envvars-prod
  2. In envvars-prod:
  • Comment the following lines
    build:
        context: .
        dockerfile: Dockerfile
    image: phccp
  • Uncomment the line image: docker.synapse.org/syn18489221/phc-collaboration-portal:develop
  1. Review the content of envvars-prod
  2. source envvars-prod && docker-compose -f docker-compose.yml up

The portal should now be accessible on https://localhost (assuming the environment variable DOMAIN=https://localhost).

Configuring the portal

Enabling authentication strategies

The portal supports local as well as Single Sign On (SSO) authentication strategies. One or more strategies can be enable or disable by setting/emptying the corresponding environment variables. See the section Authorization strategies for more detailed information.

Initialization of the database

The variable APP_INIT_DB_SEED_NAME controls if and how the database used by the portal is initialized when the server starts.

The value that this variable can take are the name of the folders listed in server/config/seeds/. The default value is APP_INIT_DB_SEED_NAME=default. This will initialize the database with the minimum objects required for the portal to work.

Currently, the codebase of the portal include a second seed named development. When setting APP_INIT_DB_SEED_NAME=development, the database is load with the default objects plus the additional data defined in the folder server/config/seeds/development. This additional data are typically suitable for demonstration purpose.

When deploying the portal for users, the portal must be initially started with APP_INIT_DB_SEED_NAME=default or any other values that you may have added. To prevent the portal to reset the database when it starts, set APP_INIT_DB_SEED_NAME=.

First login to the portal

When specifying APP_INIT_DB_SEED_NAME=default or any other non-empty seed, the database will be populated with an admin account whose credentials are defined by the environment variables APP_INIT_ADMIN_EMAIL and APP_INIT_ADMIN_PASSWORD. The purpose of this account is to allow a user to configure the portal before making it available to other users. The user can then login into the portal using either local authentication (requires AUTH_LOCAL=true) or using any of the SSO strategy configured as long as the email addressed used to login matches the email address specified by APP_INIT_ADMIN_EMAIL.

Note: When using the database seed development, a non-admin user account is created with the following credentials:

  • email: test@sagebase.org
  • password: test

Important: All the email addresses used to login into the portal must end with the domain of one of the per-registered organization. The domains supported by the database seed default are sagebase.org, sagebionetworks.org, roche.com and gene.com.

Clone this wiki locally