Creating a Dev Environment pre Docker

VIRTUALBOX / VM SETUP

1. Download Oracle VirtualBox and install it. Open it up, go to 'File > Preferences', click on 'Network', click on the 'Host-only networks' tab, and click the green plus icon to add a Host-only network. Select it and click 'Edit', and write down the IPv4 address for use in Step 3. Switch to the 'DHCP' tab, and select 'Enable Server', then enter the values below and save. Each field should contain the first three digits from the IP address and another end digit: Server Address: (first 3).100 Server Mask: 255.255.255.0 Lower Address Bound: (first 3).101 Upper Address Bound: (first 3).254

2. Download a Ubuntu 14.04 LTS ISO, then create a new VM in VirtualBox. Reserve at least 20GB of hard drive space (more if you want to be safe). After creation, right click and select Settings. Under Networking, make sure the Adapter 1's 'Attached to' to 'NAT'. Activate Adapter 2, change 'Attached to' to 'Host-only Adapter' and 'Name' to the Host-only Network name from Step 1. Under 'Advanced', change 'Promiscuous Mode' to 'Allow All'. Save and start the machine. When it asks for an ISO, point it to the ISO file you downloaded.

3. Once you are on the Ubuntu desktop, find the networking icon in the top nav bar (should be two arrows). Click 'Edit Connections, select 'Wired Connection 2' and click 'Edit'. Click the 'IPv4 Settings' tab, and change the 'Method' to 'Manual'. Set the following: Address: Choose a valid static IP you want to reference this box with. The first three digits should be the same as the ones from the Host-only Network in Step 1, and the 4th should be different. (e.g: 192.168.56.73) Netmask: 255.255.0.0 Gateway: The IP from the Host-only Network in Step 1 DNS Servers: Same IP as Gateway

4. Now go to the networking icon and select 'Edit Connections' and select 'Wired Connection 1' and 'Edit'. Click 'IPv4 Settings', change the method to 'Automatic (DHCP) addresses only'. Enter '8.8.8.8' for 'DNS Servers' and Save. Click Wired Connection 1 and Wired Connection 2 in order to reconnect with the new settings. Complete the Ubuntu install and restart.

5. [OPTIONAL] Download the Guest Additions package and mount it with Devices > Insert Guest Machine Disc. This will make the machine a little easier to use. Also set 'Devices' > 'Shared Clipboard' to 'Bidirectional'. Restart the VM afterwards.

6. Add 'local.documentcloud.org' to your HOST hosts file to reference the VM's IP chosen in Step 3. In Windows, this is under C:\Windows\System32\drivers\etc\hosts, and you will need to copy the file to another directory, edit it, and paste it back to get around read-only privileges. A sample entry, with the default local dev site URL: - '192.168.56.103 local.documentcloud.org'

6b. Add an identical 'local.documentcloud.org' entry to your GUEST VM hosts file (/etc/hosts). This is crucial for WPD and DACTYL to communicate with each other correctly over HTTP.

7. [OPTIONAL] Install Vim ('sudo apt-get install vim'), the text editor that will be referred to in the following steps. If you don't, just substitute it for another text editor.

8. Install and configure an SFTP server so that you can pull files from the host. In a command prompt in the VM:

   • 'sudo apt-get install openssh-server' (install)

9. To make the upload go smoothly:

   • 'sudo usermod -g www-data (username)' (add your user to the www-data group to make the FTP upload (and other stuff later) go smoothly)
   • 'sudo mkdir /srv/www' (create dir for upload)
   • 'sudo chown (username):www-data /srv/www' (make your user the owner of new dir)

10. Back on your host machine, install Git and clone the repo (to clone into directory 'documentcloud': 'git clone https://github.com/roberttdev/dactyl4.git documentcloud'). Get the 'secrets' directory from someone on the team and drop it under the main directory. Change the line endings config by running git config --global core.autocrlf false

11. Also clone the Document Viewer repo (to clone into directory 'document-viewer': 'git clone https://github.com/roberttdev/document-viewer4.git document-viewer').

12. Also clone the WebPlotDigitizer repo (to clone into directory 'WebPlotDigitizer': 'git clone https://github.com/roberttdev/WebPlotDigitizer.git WebPlotDigitizer').

13. Use your FTP program to connect to the VM's FTP server. Copy the entire 'documentcloud', 'document-viewer', and 'WebPlotDigitizer' project directories to be under /var/www in the VM. FTP connection info is: - Type: SFTP

    • Host: Name from Step 6, or the IP address from Step 3
    • Username/Password: username/password from setting up Ubuntu in Step 2

14. Setup DocumentCloud. In the VM, run the setup script: 'sudo bash /srv/www/documentcloud/config/server/scripts/create_environment.sh development' (WARNING: unstable! best to run each command manually! may work if you convert from Windows using dos2unix on script and all subscripts)

15. Setup WebPlotDigitizer. In the VM, run the setup script: 'sudo bash /srv/www/WebPlotDigitizer/config/create_environment.sh' (WARNING: unstable! best to run each command manually! may work if you convert from Windows using dos2unix on script and all subscripts). Try a test run of 'bash -x build.sh' to see if it executes without errors. If it does not, run dos2unix on all build scripts to clean Windows corruption out of them.

---

***RUBYMINE SETUP***

1. Install Ruby 2.1.8 and Rails 4.2 on your host machine (For Windows, install both with: https://s3.amazonaws.com/railsinstaller/Windows/railsinstaller-3.1.1.exe). Be sure to select the option to add the executables to your PATH. (Note: May be able to skip this in the future, if RM restores the ability to select a remote SDK on project creation).

2. Install RubyMine.

(DOCUMENTCLOUD)

3. Open RubyMine, and select 'Create New Project' from the splash screen. Choose 'New Application' under 'Rails'. Select the documentcloud directory and continue. It will ask you if you want to create a project from existing files, select 'Yes'.

4. Set up code deploy to VM. File > Settings > Build, Execution, Deployment > Deployment, click '+', name server and Type 'SFTP'. Click OK, then set:

   • FTP Host: hostname/IP from the VM setup
   • User name/password: username/password for logging into VM (and 'save password')
   • Root path: /srv/www/documentcloud

5. Setup DB. Click 'Database' tab on right, and click '+' > 'Data Source' > 'PostgreSQL'. Then set:

   • Host: hostname/IP from the VM setup
   • Database: dcloud_development
   • User: documentcloud
   • Password: documentcloudVirtualMachine
   • If necessary, click 'Download missing driver files'

6. Prep for developing for a Linux system. File > Settings > Editor > Code Style, set 'Line Separators' to 'Linux and Mac OSX (\n)'.

7. Point to Ruby on the VM. File > Settings > Languages & Frameworks > Ruby SDK and Gems. Click '+' and 'New Remote..'. Settings should be:

   • Deployment configuration > Deployment configuration: (server from step 4)
   • Click 'Move' for the question about server on application level
   • Ruby interpreter path: /home/(username)/.rvm/gems/ruby-2.1.8
   • Save, and when gems are done downloading, select 'Remote:' entry and click green check to make default.

8. Setup debug server launching. Go to Run > Edit Configurations and select Rails > Development: documentcloud. Change the IP address to the VM's IP address. If you'd like the browser to automatically open to the software when you launch the debug server, click 'Run browser' and change the IP address in the URL to 'local.documentcloud.org'.

9. When you want to debug, make sure 'Development: documentcloud' is the configuration and click the 'Debug' button. Connect to 'http://local.documentcloud.org:3000' to access the site.

(DOCUMENTVIEWER)

6. Select 'File' > 'New Project'. Choose 'New Application'. Select the 'document-viewer' directory. It will ask you if you want to create a project from existing files, select 'Yes'.

7. Set up code deploy to VM. File > Settings > Build, Execution, Deployment > Deployment, click '+', name server and Type 'FTP'. Click OK, then set:

   • FTP Host: hostname/IP from the VM setup
   • User name/password: username/password for logging into VM (and 'save password')
   • Root path: /srv/www/document-viewer
   • Web server Root URL: blank

(WEBPLOTDIGITIZER)

[OPTIONAL] Use a PHP-based program instead of RubyMine, since WPD is JavaScript and PHP. In PHPStorm: Create New Project, 'Web server is on remote host accessible via FTP/SFTP/FTPS', name the project and select WebPlotDigitizer directory, then: - FTP Host: hostname/IP from the VM setup - Port: 21 - User name/password: username/password for logging into VM - Root path: /srv/www/WebPlotDigitizer.

Be sure to change the line endings for Linux: File > Settings > Editor > Code Style, set 'Line Separators' to 'Linux and Mac OSX (\n)'.

---

ENVIRONMENT CLONE

If you want to pull data from another environment, you'll need to clone the DB and copy the document files. If cloning production, there is a script called restore_prod_backup.sh in the base directory that'll do most steps once you've copied a backup to the base directory and renamed it prod_bkup.backup (script is currently having problems, can run each command within manually though). You'll also need to install s3cmd (sudo apt-get install s3cmd) and copy the .s3cfg from the user home directory of the prod server to the user home directory of the VM.

Backup: On the machine hosting the DB you want to clone, run pg_dump --username "documentcloud" -F t dcloud_production > backup.sql. Copy the file to your VM's base documentcloud directory. Then run the commands:

• sudo service postgres restart (drop connections)
• sudo -u postgres dropdb dcloud_development
• sudo -u postgres createdb dcloud_development
• sudo -u postgres pg_restore -d dcloud_development prod_bkup.backup
• bundle exec rake db: migrate

Document Files: If you're downloading from S3, they're stored in the 'documents' folder. If you're copying from a non-S3 enviroment, they're in 'public/asset_store'. They need to be copied to 'public/asset_store' in the Dev VM.