Closes #18780: External database configuration (#18912)

Author: jeremystretch

docs/configuration/required-parameters.md

@@ -25,7 +25,30 @@ ALLOWED_HOSTS = ['*']
 
 ## DATABASE
 
-NetBox requires access to a PostgreSQL 14 or later database service to store data. This service can run locally on the NetBox server or on a remote system. The following parameters must be defined within the `DATABASE` dictionary:
+!!! warning "Legacy Configuration Parameter"
+    The `DATABASE` configuration parameter is deprecated and will be removed in a future release. Users are advised to adopt the new `DATABASES` (plural) parameter, which allows for the configuration of multiple databases.
+
+See the [`DATABASES`](#databases) configuration below for usage.
+
+---
+
+## DATABASES
+
+!!! info "This parameter was introduced in NetBox v4.3."
+
+NetBox requires access to a PostgreSQL 14 or later database service to store data. This service can run locally on the NetBox server or on a remote system. Databases are defined as named dictionaries:
+
+```python
+DATABASES = {
+    'default': {...},
+    'external1': {...},
+    'external2': {...},
+}
+```
+
+NetBox itself requires only that a `default` database is defined. However, certain plugins may require the configuration of additional databases. (Consider also configuring the [`DATABASE_ROUTERS`](./system.md#database_routers) parameter when multiple databases are in use.)
+
+The following parameters must be defined for each database:
 
 * `NAME` - Database name
 * `USER` - PostgreSQL username
@@ -38,22 +61,24 @@ NetBox requires access to a PostgreSQL 14 or later database service to store dat
 Example:
 
 ```python
-DATABASE = {
-    'ENGINE': 'django.db.backends.postgresql',
-    'NAME': 'netbox',               # Database name
-    'USER': 'netbox',               # PostgreSQL username
-    'PASSWORD': 'J5brHrAXFLQSif0K', # PostgreSQL password
-    'HOST': 'localhost',            # Database server
-    'PORT': '',                     # Database port (leave blank for default)
-    'CONN_MAX_AGE': 300,            # Max database connection age
+DATABASES = {
+    'default': {
+        'ENGINE': 'django.db.backends.postgresql',
+        'NAME': 'netbox',               # Database name
+        'USER': 'netbox',               # PostgreSQL username
+        'PASSWORD': 'J5brHrAXFLQSif0K', # PostgreSQL password
+        'HOST': 'localhost',            # Database server
+        'PORT': '',                     # Database port (leave blank for default)
+        'CONN_MAX_AGE': 300,            # Max database connection age
+    }
 }
 ```
 
 !!! note
     NetBox supports all PostgreSQL database options supported by the underlying Django framework. For a complete list of available parameters, please see [the Django documentation](https://docs.djangoproject.com/en/stable/ref/settings/#databases).
 
 !!! warning
-    Make sure to use a PostgreSQL-compatible backend for the ENGINE setting. If you don't specify an ENGINE, the default will be django.db.backends.postgresql.
+    The `ENGINE` parameter must specify a PostgreSQL-compatible database backend. If not defined, the default engine `django.db.backends.postgresql` will be used.
 
 ---
 

docs/configuration/system.md

@@ -12,6 +12,14 @@ BASE_PATH = 'netbox/'
 
 ---
 
+## DATABASE_ROUTERS
+
+Default: `[]` (empty list)
+
+An iterable of [database routers](https://docs.djangoproject.com/en/stable/topics/db/multi-db/) to use for automatically selecting the appropriate database(s) for a query. This is useful only when [multiple databases](./required-parameters.md#databases) have been configured.
+
+---
+
 ## DEFAULT_LANGUAGE
 
 Default: `en-us` (US English)

docs/development/getting-started.md

@@ -115,7 +115,7 @@ You may also need to set up the yarn packages as shown in the [Web UI Developmen
 Within the `netbox/netbox/` directory, copy `configuration_example.py` to `configuration.py` and update the following parameters:
 
 * `ALLOWED_HOSTS`: This can be set to `['*']` for development purposes
-* `DATABASE`: PostgreSQL database connection parameters
+* `DATABASES`: PostgreSQL database connection parameters
 * `REDIS`: Redis configuration (if different from the defaults)
 * `SECRET_KEY`: Set to a random string (use `generate_secret_key.py` in the parent directory to generate a suitable key)
 * `DEBUG`: Set to `True`

docs/installation/3-netbox.md

@@ -128,7 +128,7 @@ sudo cp configuration_example.py configuration.py
 Open `configuration.py` with your preferred editor to begin configuring NetBox. NetBox offers [many configuration parameters](../configuration/index.md), but only the following four are required for new installations:
 
 * `ALLOWED_HOSTS`
-* `DATABASE`
+* `DATABASES` (or `DATABASE`)
 * `REDIS`
 * `SECRET_KEY`
 
@@ -146,18 +146,22 @@ If you are not yet sure what the domain name and/or IP address of the NetBox ins
 ALLOWED_HOSTS = ['*']
 ```
 
-### DATABASE
+### DATABASES
 
-This parameter holds the database configuration details. You must define the username and password used when you configured PostgreSQL. If the service is running on a remote host, update the `HOST` and `PORT` parameters accordingly. See the [configuration documentation](../configuration/required-parameters.md#database) for more detail on individual parameters.
+This parameter holds the PostgreSQL database configuration details. The default database must be defined; additional databases may be defined as needed e.g. by plugins.
+
+A username and password must be defined for the default database. If the service is running on a remote host, update the `HOST` and `PORT` parameters accordingly. See the [configuration documentation](../configuration/required-parameters.md#databases) for more detail on individual parameters.
 
 ```python
-DATABASE = {
-    'NAME': 'netbox',               # Database name
-    'USER': 'netbox',               # PostgreSQL username
-    'PASSWORD': 'J5brHrAXFLQSif0K', # PostgreSQL password
-    'HOST': 'localhost',            # Database server
-    'PORT': '',                     # Database port (leave blank for default)
-    'CONN_MAX_AGE': 300,            # Max database connection age (seconds)
+DATABASES = {
+    'default': {
+        'NAME': 'netbox',               # Database name
+        'USER': 'netbox',               # PostgreSQL username
+        'PASSWORD': 'J5brHrAXFLQSif0K', # PostgreSQL password
+        'HOST': 'localhost',            # Database server
+        'PORT': '',                     # Database port (leave blank for default)
+        'CONN_MAX_AGE': 300,            # Max database connection age (seconds)
+    }
 }
 ```
 

netbox/netbox/configuration_example.py

@@ -12,14 +12,16 @@
 
 # PostgreSQL database configuration. See the Django documentation for a complete list of available parameters:
 #   https://docs.djangoproject.com/en/stable/ref/settings/#databases
-DATABASE = {
-    'ENGINE': 'django.db.backends.postgresql',  # Database engine
-    'NAME': 'netbox',         # Database name
-    'USER': '',               # PostgreSQL username
-    'PASSWORD': '',           # PostgreSQL password
-    'HOST': 'localhost',      # Database server
-    'PORT': '',               # Database port (leave blank for default)
-    'CONN_MAX_AGE': 300,      # Max database connection age
+DATABASES = {
+    'default': {
+        'ENGINE': 'django.db.backends.postgresql',  # Database engine
+        'NAME': 'netbox',         # Database name
+        'USER': '',               # PostgreSQL username
+        'PASSWORD': '',           # PostgreSQL password
+        'HOST': 'localhost',      # Database server
+        'PORT': '',               # Database port (leave blank for default)
+        'CONN_MAX_AGE': 300,      # Max database connection age
+    }
 }
 
 # Redis database settings. Redis is used for caching and for queuing background tasks such as webhook events. A separate

netbox/netbox/configuration_testing.py

@@ -5,13 +5,15 @@
 
 ALLOWED_HOSTS = ['*']
 
-DATABASE = {
-    'NAME': 'netbox',
-    'USER': 'netbox',
-    'PASSWORD': 'netbox',
-    'HOST': 'localhost',
-    'PORT': '',
-    'CONN_MAX_AGE': 300,
+DATABASES = {
+    'default': {
+        'NAME': 'netbox',
+        'USER': 'netbox',
+        'PASSWORD': 'netbox',
+        'HOST': 'localhost',
+        'PORT': '',
+        'CONN_MAX_AGE': 300,
+    }
 }
 
 PLUGINS = [

netbox/netbox/settings.py

@@ -53,10 +53,14 @@
         )
     raise
 
-# Check for missing required configuration parameters
-for parameter in ('ALLOWED_HOSTS', 'DATABASE', 'SECRET_KEY', 'REDIS'):
+# Check for missing/conflicting required configuration parameters
+for parameter in ('ALLOWED_HOSTS', 'SECRET_KEY', 'REDIS'):
     if not hasattr(configuration, parameter):
         raise ImproperlyConfigured(f"Required parameter {parameter} is missing from configuration.")
+if not hasattr(configuration, 'DATABASE') and not hasattr(configuration, 'DATABASES'):
+    raise ImproperlyConfigured("The database configuration must be defined using DATABASE or DATABASES.")
+elif hasattr(configuration, 'DATABASE') and hasattr(configuration, 'DATABASES'):
+    raise ImproperlyConfigured("DATABASE and DATABASES may not be set together. The use of DATABASES is encouraged.")
 
 # Set static config parameters
 ADMINS = getattr(configuration, 'ADMINS', [])
@@ -84,7 +88,9 @@
 CSRF_COOKIE_SECURE = getattr(configuration, 'CSRF_COOKIE_SECURE', False)
 CSRF_TRUSTED_ORIGINS = getattr(configuration, 'CSRF_TRUSTED_ORIGINS', [])
 DATA_UPLOAD_MAX_MEMORY_SIZE = getattr(configuration, 'DATA_UPLOAD_MAX_MEMORY_SIZE', 2621440)
-DATABASE = getattr(configuration, 'DATABASE')  # Required
+DATABASE = getattr(configuration, 'DATABASE', None)  # Legacy DB definition
+DATABASE_ROUTERS = getattr(configuration, 'DATABASE_ROUTERS', [])
+DATABASES = getattr(configuration, 'DATABASES', {'default': DATABASE})
 DEBUG = getattr(configuration, 'DEBUG', False)
 DEFAULT_DASHBOARD = getattr(configuration, 'DEFAULT_DASHBOARD', None)
 DEFAULT_PERMISSIONS = getattr(configuration, 'DEFAULT_PERMISSIONS', {
@@ -220,17 +226,15 @@
 # Database
 #
 
-# Set the database engine
-if 'ENGINE' not in DATABASE:
-    if METRICS_ENABLED:
-        DATABASE.update({'ENGINE': 'django_prometheus.db.backends.postgresql'})
-    else:
-        DATABASE.update({'ENGINE': 'django.db.backends.postgresql'})
+# Verify that a default database has been configured
+if 'default' not in DATABASES:
+    raise ImproperlyConfigured("No default database has been configured.")
 
-# Define the DATABASES setting for Django
-DATABASES = {
-    'default': DATABASE,
-}
+# Set the database engine
+if 'ENGINE' not in DATABASES['default']:
+    DATABASES['default'].update({
+        'ENGINE': 'django_prometheus.db.backends.postgresql' if METRICS_ENABLED else 'django.db.backends.postgresql'
+    })
 
 
 #