Enable and use ElasticSearch basic authentication

.gitignore

@@ -15,9 +15,11 @@
 /*/thehive/logs/*
 
 # Ignore custom configuration files for TheHive
+/*/thehive/config/index.conf
 /*/thehive/config/secret.conf
 
 # Ignore custom configuration files for Cortex
+/*/cortex/config/index.conf
 /*/cortex/config/secret.conf
 
 # Keep track of .gitkeep files to guarantee folder permissions are correct
@@ -30,4 +32,4 @@
 # Ignore server keys/certs
 /**/server.key
 /**/server.crt
-/**/ca.pem
+/**/ca.pem

README.md

@@ -6,6 +6,26 @@
 
 This repository offers various Docker Compose profiles to set up TheHive and Cortex for different use cases, including both testing and production environments. These profiles are designed to simplify deployment and ensure optimal performance based on your infrastructure requirements.
 
+## Upgrade notes
+
+> [!CAUTION]
+> ElasticSearch authentication was added [by this PR](https://github.com/StrangeBeeCorp/docker/pull/14), follow these instructions to implement it in older deployments
+
+To implement ElasticSearch authentication, we need to recreate the `.env` file and create new `index.conf`.
+This is done automatically by doing the following commands:
+```bash
+# Pull the latest version of this codebase
+git pull
+
+# Go to the relevant profile (e.g. "testing" here)
+cd ./testing
+
+# Run the "init.sh" script
+./scripts/init.sh
+```
+
+A password for ElasticSearch will be created and propagated to TheHive and Cortex.
+
 ## Available Deployment Profiles
 
 - [Testing environment](./testing/): Deploys TheHive and Cortex on a single server for testing purposes. Requirements: 8 GB RAM, 4 vCPU.
@@ -22,4 +42,4 @@ These Docker Compose profiles include scripts to help you manage and optimize va
 - Docker engine `v23.0.15` and later ([install instructions](https://docs.docker.com/engine/install/))
 - Docker compose plugin `v2.20.2` and later ([install instructions](https://docs.docker.com/compose/install/))
 - jq [https://jqlang.github.io/jq/](https://jqlang.github.io/jq/)
-- Current user should have at least sudo permissions 
+- Current user should have at least sudo permissions

prod1-cortex/README.md

@@ -28,15 +28,15 @@ docker compose version
 
 ## Content of the application stack
 
-This *testing* Docker Compose file deploys the following components:
+This *production* Docker Compose file deploys the following components:
 
 * Elasticsearch: Serves as the database for Cortex
 * Cortex: Analyzers and Responders engine
 * Nginx: Deployed as an HTTPS reverse proxy
 
 ### Configuration and data files
 
-Each container has as dedicated folder for configuration, data and log files. 
+Each container has as dedicated folder for configuration, data and log files.
 
 ```bash
 .
@@ -64,22 +64,23 @@ elasticsearch
 * **./elasticsearch/logs**: the log files
 
 > [!NOTE]
-> These folders should not be manually modified 
+> These folders should not be manually modified
 
 #### Cortex
 
 ```bash
 cortex
 ├── config
 │   ├── application.conf
+│   ├── index.conf.template
 │   ├── logback.xml
 │   └── secret.conf.template
 ├── cortex-jobs
 ├── logs
 └── neurons
 ```
 
-* **./cortex/config**: configuration files. `secret.conf` is generated automatically when using our init script.
+* **./cortex/config**: configuration files. `index.conf` and `secret.conf` are generated automatically when using our init script.
 * **./cortex/cortex-jobs**: temprary data storage for Analyzers and Responders
 * **./cortex/logs**: Cortex log files
 * **./cortex/neurons**: Folder dedicated to custom Analyzers and Responders
@@ -104,9 +105,9 @@ nginx
 
 #### Certificates
 
-This foler is empty. By default, the application stack is initialised with self-signed certificates. 
+This folder is empty. By default, the application stack is initialised with self-signed certificates.
 
-If you want to use your own certificates, like one signed by an internal authority, create following files - ensure to use the filenames written - : 
+If you want to use your own certificates, like one signed by an internal authority, create following files - ensure to use the filenames written - :
 
 ```bash
 certificates
@@ -139,18 +140,18 @@ The application stack includes several utility scripts:
 
 The application will run under the user account and group that executes the init script.
 
-Run the *init.sh* script: 
+Run the *init.sh* script:
 
 ```bash
 bash ./scripts/init.sh
 ```
 
-This script wil perform following actions: 
+This script wil perform following actions:
 
 * Prompts for a service name to include in the Nginx server certificate.
-* Initializes the `secret.conf` files for Cortex.
+* Initializes the `index.conf` and `secret.conf` files for Cortex.
 * Generates self-signed certificate none is found in `./certificates`
-* Creates a `.env` file will user/group information and other application settings
+* Creates a `.env` file with user/group information and other application settings
 * Verifies file and folder permissions.
 
 
@@ -160,38 +161,38 @@ This script wil perform following actions:
 docker compose up
 ```
 
-or 
+or
 
 ```bash
 docker compose up -d
 ```
 
 ## Access to the applications
 
-Open your browser, and navigate to: 
+Open your browser, and navigate to:
 
 * `https://HOSTNAME_OR_IP` to connect to Cortex
 
 
 ## Additional content
 
-Multiple scripts are also provided to help managing and testing the applications: 
+Multiple scripts are also provided to help managing and testing the applications:
 
 ### Reset your environment
 
-Run the following script to delete all data in the *testing* environment: 
+Run the following script to delete all data in the *production* environment:
 
 ```bash
 bash ./scripts/reset.sh
 ```
 
 > [!CAUTION]
-> This scripts deletes all data and containers. 
+> This scripts deletes all data and containers.
+
+Run the *init.sh* script to reload a new *production* instance.
 
-Run the *init.sh* script to reload a new *production* instance. 
-
 
 ### Backup / Restore
 
 This profile includes two utility scripts to assist with performing cold backups and restores. You can find these scripts here: [./scripts/backup.sh](./scripts/backup.sh) and [./scripts/restore.sh](./scripts/restore.sh)
-For detailed information about backup and restore strategies and processes, please refer to the [dedicated documentation](https://docs.strangebee.com/thehive/operations/backup-restore/overview/).
+For detailed information about backup and restore strategies and processes, please refer to the [dedicated documentation](https://docs.strangebee.com/thehive/operations/backup-restore/overview/).

prod1-cortex/cortex/config/application.conf

@@ -1,5 +1,9 @@
+# SECRET CONFIGURATION
 include "/etc/cortex/secret.conf"
 
+# INDEX DATABASE CONFIGURATION
+include "/etc/cortex/index.conf"
+
 play.http.context="/"
 play.http.parser.maxMemoryBuffer=1M
 play.http.parser.maxDiskBuffer=1G
@@ -12,11 +16,9 @@ cache.job = 10 minutes
 # docker.container.extraHosts=["hostname:ip,hostname:ip"]
 
 auth {
-        provider = [local]
-        ad {
-        }
-        ldap {
-        }
+  provider = [local]
+  ad {}
+  ldap {}
 }
 
 analyzer {
@@ -44,4 +46,3 @@ responder {
     parallelism-max = 4
   }
 }
-

prod1-cortex/cortex/config/index.conf.template

@@ -0,0 +1,5 @@
+search {
+  index = cortex
+  user = "elastic"
+  password = "###CHANGEME_ELASTICSEARCH_PASSWORD###"
+}

prod1-cortex/docker-compose.yml

@@ -12,7 +12,8 @@ services:
       - thread_pool.search.queue_size=100000
       - thread_pool.write.queue_size=100000
       - bootstrap.memory_lock=true
-      - xpack.security.enabled=false
+      - xpack.security.enabled=true
+      - ELASTIC_PASSWORD=${elasticsearch_password} # Password for "elastic" default user
       - ES_JAVA_OPTS=-Xms6G -Xmx6G
     volumes:
       - ./elasticsearch/data:/usr/share/elasticsearch/data
@@ -30,7 +31,7 @@ services:
     networks:
       - cortex-network
     healthcheck:
-      test: ["CMD-SHELL", "curl -s -f elasticsearch:9200/_cat/health || exit 1"]
+      test: ["CMD-SHELL", "curl -f -s -u 'elastic:${elasticsearch_password}' http://elasticsearch:9200/_cat/health || exit 1"]
       start_period: 40s
       interval: 2s
       timeout: 1s

prod1-cortex/dot.env.template

@@ -4,5 +4,8 @@ UID=XXXX
 ## Replace XXXX with the output of the command "id -g"
 GID=XXXX
 
+# ElasticSearch password
+elasticsearch_password = '###CHANGEME_ELASTICSEARCH_PASSWORD###'
+
 # Cortex specific configuration
 cortex_docker_job_directory=${PWD}/cortex/cortex-jobs

prod1-cortex/scripts/init.sh

@@ -18,7 +18,17 @@ SERVICE_HOSTNAME=${choice:-${SYSTEM_HOSTNAME}}
 
 
 init() {
+    ELASTICSEARCH_PASSWORD=$(cat /dev/urandom | LC_CTYPE=C tr -dc '[:alnum:]' | fold -w 64 | head -n 1)
+
     ## INIT CORTEX CONFIGURATION
+    CORTEXINDEXFILE="./cortex/config/index.conf"
+    CORTEXINDEXFILETEMPLATE="./cortex/config/index.conf.template"
+    if [ -f ${CORTEXINDEXFILE} ]
+    then
+        rm -f ${CORTEXINDEXFILE}
+    fi
+    sed -e "s/###CHANGEME_ELASTICSEARCH_PASSWORD###/$ELASTICSEARCH_PASSWORD/g" < $CORTEXINDEXFILETEMPLATE > $CORTEXINDEXFILE
+
     CORTEXSECRETFILE="./cortex/config/secret.conf"
     if [ ! -f ${CORTEXSECRETFILE} ]
     then
@@ -32,15 +42,19 @@ _EOF_
 
     ## CREATE .env FILE
     ENVFILE="./.env"
-    if [ ! -f ${ENVFILE} ]
+    if [ -f ${ENVFILE} ]
     then
-        CURRENT_USER_ID=$(id -u)
-        CURRENT_GROUP_ID=$(id -g)
-        cat ../versions.env dot.env.template > .env
-        define_hostname # Ask user for service hostname
-        check_user_certificates ${SYSTEM_HOSTNAME}
-        # bash $(dirname $0)/generate_certs.sh ${SYSTEM_HOSTNAME} # Generate Nginx self-signed certificates if no certificate is installed.
-        cat >> ${ENVFILE} << _EOF_
+        rm -f ${ENVFILE}
+    fi
+    CURRENT_USER_ID=$(id -u)
+    CURRENT_GROUP_ID=$(id -g)
+    sed -e "s/###CHANGEME_ELASTICSEARCH_PASSWORD###/$ELASTICSEARCH_PASSWORD/g" < ./dot.env.template > $ENVFILE
+    cat ../versions.env >> .env
+    # Ask user for service hostname
+    define_hostname
+    check_user_certificates ${SYSTEM_HOSTNAME}
+    # bash $(dirname $0)/generate_certs.sh ${SYSTEM_HOSTNAME} # Generate Nginx self-signed certificates if no certificate is installed.
+    cat >> ${ENVFILE} << _EOF_
 ## CONFIGURATION AUTOMATICALLY ADDED BY .scripts/init.sh PROGRAM.
 # System variables
 UID=${CURRENT_USER_ID}
@@ -51,12 +65,6 @@ nginx_server_name="${SERVICE_HOSTNAME}"
 nginx_ssl_trusted_certificate="${NGINX_SSL_TRUSTED_CERTIFICATE_CONFIG}"
 _EOF_
 
-    else
-        STATUS=1
-        warning "${ENVFILE} file already exists and has not been modified."
-        exit 0
-    fi
-
     if [ ${STATUS} == 0 ]
     then
         success "Initialisation completed."

prod1-cortex/scripts/reset.sh

@@ -1,6 +1,6 @@
 #! /usr/bin/env bash
 
-## This program remove all data and delete all files created by init.sh script. Once executed successfully, the folder is clean with no data. 
+## This program remove all data and delete all files created by init.sh script. Once executed successfully, the folder is clean with no data.
 
 source $(dirname $0)/output.sh  # used to display output
 
@@ -22,10 +22,10 @@ then
   done
   success "All data removed."
 
-  ## DELETE secret.conf FILES
-  rm ./thehive/config/secret.conf
+  ## DELETE index.conf and secret.conf FILES
+  rm ./cortex/config/index.conf
   rm ./cortex/config/secret.conf
-  success "secret files deleted."
+  success "index and secret files deleted."
 
   ## DELETE cert files
   rm -rf ./nginx/certs/*
@@ -46,7 +46,7 @@ then
       sudo chown ${CURRENT_USER_ID}:${CURRENT_GROUP_ID} "${line}"
       success "Ownership updated for ${line}"
       done
-    
+
     [[ $? -ne 0 ]] && info "Run this command with root privileges to complete the reset process:\n
     # find . ! -user ${CURRENT_USER_ID} -o ! -group ${CURRENT_GROUP_ID} -exec chown ${CURRENT_USER_ID}:${CURRENT_GROUP_ID} {} \; "
   fi
@@ -56,4 +56,4 @@ then
 
 else
   exit 0
-fi
+fi