Skip to content

Installation and running

Jump to bottom
Václav Bartoš edited this page Jul 9, 2021 · 7 revisions

Installation and running

The NERD system is designed mainly for CentOS 7 system. It can probably run on other systems as well, but it would need some changes in installation scripts and configuration, which is not tested nor documented.

Vargant (for development)

For development and testing, it is recommended to use Vagrant with the provided Vagrantfile. It creates a VM with CentOS 7, installs and configures all the dependencies and the NERD system itself. The only thing needed to do manually is to configure connection to Warden and keys for some external services, such as Shodan (but that's usually not needed for a quick test). Then, run NERD backend by starting its supervisor (systemctl start nerd-supervisor).

Assuming you have Vagrant installed, all you need to do is:

cd <root of NERD git repository>
vagrant up   # create and run the VM
# wait, it may take quite a long time to install everything
# read the instructions printed at the end
vagrant ssh  # SSH into the VM

sudo systemctl start nerd-supervisor

There are several web interfaces running in the VM, available from the host:

  • NERD web: http://localhost:8080/nerd/
  • Supervisor GUI: http://localhost:9001/
  • RabbitMQ management interface: http://localhost:15672/

Inside Vagrant VM, the host's current directory (i.e. the git repository) is mounted to /vagrant/.

To do some data processing, you need to copy some .idea files into /data/warden_filer/warden_receiver/incoming. Or configure warden_filer to read data from a Warden server.

The NERD web interface is run automatically as part of Apache via WSGI. To reload it (e.g. after some files are changed), run sudo systemctl reload httpd.

The backend components can be controlled via nerdctl (an alias for supervisorctl -c /etc/nerd/supervisord.conf). Logs can be found in /var/log/nerd/.

Note: The VM sometimes hangs under high load, e.g. when processing all data from CESNET's Warden. It seems related to kernel and network interface, but I don't know exact reason or solution. Anyway, the Vagrant VM is intended only for quick testing of new features, not for continuous running under high load.

Full installation

There an installation script that installs and configures almost everything automatically.

  1. Set up a server with clear CentOS 7 installation (recommended at least 4 CPU, 8GB RAM, more RAM will help).
  2. Download the installation script: install_centos7.sh.
  3. Run the script as root. It'll download the latest version of NERD from git repository, install all requirements and configures everything which is possible to configure automatically.
  4. Follow the instructions printed by the script to finish configuration.
  5. Configure Warden client (you first need to register your client at Warden server to be able to receive data).
  6. Create user accounts for web interface / API (only local accounts are supported, shibboleth (federated login) must be installed and configured manually if needed).
  7. Run NERD backend sudo systemctl start nerd-supervisor
  8. Check frontend at https://<server_address>/nerd/

Installation paths

  • All the code (programs, web) is installed into /nerd/.
  • Configuration is located in /etc/nerd/.
  • Logs go to /var/log/nerd/.
  • Various data, such as blacklists, geolocation database or IDEA messages from warden, are located in subdirectories under /data/.

Clone this wiki locally