Skip to content

Latest commit

 

History

History
115 lines (103 loc) · 5.82 KB

VM_SETUP.md

File metadata and controls

115 lines (103 loc) · 5.82 KB
Raw

Create VM

In Azure

  1. When creating VM, enable login using Azure Active Directory
  2. Restrict SSH port to only the jumpbox-ip
  3. Open ports 80 (http), 443 (https) and 3099 (webhook)
  4. Set the DNS Name Label on the IP Address attached to the VM

In GitHub

  1. Create web-hook to listen on http://ip-address:3099/
  2. Set secret for web-hook access (for this step)

In VM

  1. User Management

    1. Create user group - sudo groupadd ibf-users
    2. Add ibf-user to group - sudo usermod -a -G ibf-users ibf-user
    3. Add users to group - sudo usermod -a -G ibf-users grahman@rodekruis.nl
      1. Command to verify group members - grep ibf-users /etc/group
    4. Change access of shared directory - /home/ibf-user
      1. chgrp -Rf ibf-users /home/ibf-user
      2. sudo chown -R ibf-user:ibf-users /home/ibf-user
      3. sudo chmod -R 775 /home/ibf-user
      4. Re-login to verify if you have access by running touch /home/ibf-user
    5. Add the following lines to /etc/sudoers
    # Allow members of group ibf-users to execute systemctl daemon-reload
%ibf-users ALL=NOPASSWD: /bin/systemctl daemon-reload

# Allow members of group ibf-users to execute service webhook restart
%ibf-users ALL=NOPASSWD: /usr/sbin/service webhook restart

  2. Install Software

    1. NodeJS Source
      1. curl -sL https://deb.nodesource.com/setup_14.x | sudo -E bash -
      2. sudo apt-get install -y nodejs
      3. Verification - node -v
    2. Docker Source
      1. sudo apt-get remove docker docker-engine docker.io containerd runc
      2. sudo apt-get update
      3. sudo apt-get install apt-transport-https ca-certificates curl gnupg-agent software-properties-common
      4. curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
      5. sudo apt-key fingerprint 0EBFCD88
      6. sudo add-apt-repository \ "deb [arch=amd64] https://download.docker.com/linux/ubuntu \ $(lsb_release -cs) \ stable"
      7. sudo apt-get update
      8. sudo apt-get install docker-ce docker-ce-cli containerd.io
      9. Allow users to access docker commands
        1. sudo usermod -aG docker grahman@rodekruis.nl
        2. Verification - grep docker /etc/group
      10. Verification - docker -v
    3. Docker Compose Source
      1. sudo curl -L "[https://github.com/docker/compose/releases/download/1.27.4/docker-compose-$](https://github.com/docker/compose/releases/download/1.27.4/docker-compose-$)(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
      2. sudo chmod +x /usr/local/bin/docker-compose
      3. Verification - docker-compose -v
    4. Webhook Related
      1. sudo apt install postgresql-client-common postgresql-client

  3. Setup IBF-system

    1. Setup GIT
      1. git clone https://github.com/rodekruis/IBF-system.git
      2. Set the repo config to allow group access - git config core.sharedRepository group Source
      3. sudo chgrp -R ibf-users /home/ibf-user/IBF-system Source
      4. sudo chmod -R g+rwX /home/ibf-user/IBF-system Source
    2. Setup Environment Variables
      1. Create services/IBF-pipeline/pipeline/secrets.py
        1. cp /home/ibf-user/IBF-system/services/IBF-pipeline/pipeline/secrets.py.template /home/ibf-user/IBF-system/services/IBF-pipeline/pipeline/secrets.py
        2. Set the appropriate values in the /home/ibf-user/IBF-system/services/IBF-pipeline/pipeline/secrets.py file
      2. Create /home/ibf-user/IBF-system/.env
        1. cp /home/ibf-user/IBF-system/example.env /home/ibf-user/IBF-system/.env
        2. Set the appropriate values in the .env file
        3. Load the .env vars by source /home/ibf-user/IBF-system/.env
        4. Test if the vars were loaded correctly echo $NODE_ENV
    3. Setup web-hook
      1. sudo cp webhook.service /etc/systemd/system/
      2. Set GITHUB_WEBHOOK_SECRET value in /etc/systemd/system/webhook.service
      3. Verification - ls /etc/systemd/system/
      4. In /home/ibf-user/IBF-system - npm install github-webhook-handler
      5. sudo service webhook start
      6. Verification - sudo service webhook status
    4. . tools/deploy.sh

  4. Load base data for pipeline

    1. Download these 2 data files
    2. Unzip the files using apt install unzip and unzip data.zip,
      1. geodata.zip into services/IBF-pipeline/geoserver/geodata/
      2. data.zip into services/IBF-pipeline/pipeline/data/
    3. You can verify that the pipeline runs correctly by going into the container (docker-compose exec ibf-pipeline bash) and running python3 runPipeline.py
    4. The pipeline will run automatically every day through a cron job, which requires no additional set-up.

  5. Verify if the geoserver is set up correctly.

    1. Check if the Geoserver contains necessary layers.

Useful Commands / Tools

  1. Delete user from group? sudo gpasswd -d grahman@rodekruis.nl ibf-users
  2. Secret Generator