Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 68 additions & 35 deletions packages/oracle/_dev/build/docs/README.md
Original file line number Diff line number Diff line change
@@ -1,41 +1,67 @@
# Oracle Integration

This integration is for ingesting Audit Trail logs and fetching performance, tablespace and sysmetric metrics from Oracle Databases.
## Overview

The integration expects an *.aud audit file that is generated from Oracle Databases by default. If this has been disabled then please see the [Oracle Database Audit Trail Documentation](https://docs.oracle.com/en/database/oracle/oracle-database/19/dbseg/introduction-to-auditing.html#GUID-8D96829C-9151-4FA4-BED9-831D088F12FF).
Use the Oracle integration to ingest Oracle Database audit logs and collect database metrics for operational visibility and troubleshooting.

### Requirements
### Compatibility

Connectivity to Oracle can be facilitated in two ways either by using official Oracle libraries or by using a JDBC driver. Facilitation of the connectivity using JDBC is not supported currently with Metricbeat. Connectivity can be facilitated using Oracle libraries and the detailed steps to do the same are mentioned below.
This integration has been tested with Oracle Database 19c, and should work for 18c as well though it has not been tested.

## What data does this integration collect?

This integration collects:
- Audit logs from Oracle Database audit trail files.
- Metrics covering tablespace usage, system metrics, memory, system statistics, and performance.

:::{note}
The Oracle integration expects an `*.aud` audit file that is generated from Oracle Databases by default. If disabled, refer to the [Oracle Database Audit Trail Documentation](https://docs.oracle.com/en/database/oracle/oracle-database/19/dbseg/introduction-to-auditing.html#GUID-8D96829C-9151-4FA4-BED9-831D088F12FF).
:::

## What do I need to use this integration?

Connectivity to Oracle can be facilitated in two ways: by using official Oracle libraries, or by using a JDBC driver. Connectivity through JDBC is currently not supported with Metricbeat.

#### Oracle Database Connection Pre-requisites
To get connected with the Oracle Database, you must set the following environment variables: `ORACLE_SID`, `ORACLE_BASE`, `ORACLE_HOME`.

To get connected with the Oracle Database ORACLE_SID, ORACLE_BASE, ORACLE_HOME environment variables should be set.
For example, let’s consider Oracle Database 21c installation using RPM manually by following the [Oracle Installation instructions](https://docs.oracle.com/en/database/oracle/oracle-database/21/ladbi/running-rpm-packages-to-install-oracle-database.html). The environment variables should be set as follows:

- `ORACLE_SID=ORCLCDB`
- `ORACLE_BASE=/opt/oracle/oradata`
- `ORACLE_HOME=/opt/oracle/product/21c/dbhome_1`

For example: Let’s consider Oracle Database 21c installation using RPM manually by following the [Oracle Installation instructions](https://docs.oracle.com/en/database/oracle/oracle-database/21/ladbi/running-rpm-packages-to-install-oracle-database.html). Environment variables should be set as follows:
`ORACLE_SID=ORCLCDB`
`ORACLE_BASE=/opt/oracle/oradata`
`ORACLE_HOME=/opt/oracle/product/21c/dbhome_1`
Also, add `$ORACLE_HOME/bin` to the `PATH` environment variable.

#### Oracle Instant Client
## How do I deploy this integration?

Oracle Instant Client enables development and deployment of applications that connect to Oracle Database. The Instant Client libraries provide the necessary network connectivity and advanced data features to make full use of Oracle Database. If you have OCI Oracle server which comes with these libraries pre-installed, you don't need a separate client installation.
1. In Kibana, go to **Management > Integrations**.
2. In the "Search for integrations" search bar, type **Oracle**.
3. Click the **Oracle** integration from the search results.
4. Click the **Oracle** button to add the integration.
5. Configure:
- Logs (audit trail): point the Elastic Agent filestream input to the Oracle audit file location (*.aud).
- Metrics (SQL input): provide an Oracle DSN.
6. Click **Save and Continue** to save the integration.

The OCI library install few Client Shared Libraries that must be referenced on the machine where Metricbeat is installed. Please follow the [Oracle Client Installation link](https://docs.oracle.com/en/database/oracle/oracle-database/21/lacli/install-instant-client-using-zip.html#GUID-D3DCB4FB-D3CA-4C25-BE48-3A1FB5A22E84) link for OCI Instant Client set up. The OCI Instant Client is available with the Oracle Universal Installer, RPM file or ZIP file. Download links can be found at the [Oracle Instant Client Download page](https://www.oracle.com/database/technologies/instant-client/downloads.html).
## Oracle Instant Client

If Elastic Agent is running as a systemd service and not using `ldconfig` is an option, to update the links to the shared libraries, you can use the `LD_LIBRARY_PATH` environment variable instead. Follow these steps to ensure Elastic Agent and its spawned processes respect the `LD_LIBRARY_PATH` environment variable.
Oracle Instant Client (OCI) enables development and deployment of applications that connect to Oracle Database. The Instant Client libraries provide the necessary network connectivity and advanced data features to make full use of Oracle Database. If you have OCI Oracle server which comes with these libraries pre-installed, you don't need a separate client installation.

> Prerequisites: Ensure that you have administrative privileges to modify the Elastic Agent systemd service configuration.
The OCI library installs few Client Shared Libraries that must be referenced on the machine where Metricbeat is installed. To set up the OCI Instant Client, follow the [Oracle Client Installation](https://docs.oracle.com/en/database/oracle/oracle-database/21/lacli/install-instant-client-using-zip.html#GUID-D3DCB4FB-D3CA-4C25-BE48-3A1FB5A22E84) instructions. The OCI Instant Client is available with the Oracle Universal Installer, RPM file or ZIP file that you can download from the [Oracle Instant Client Download](https://www.oracle.com/database/technologies/instant-client/downloads.html) page.

If Elastic Agent is running as a systemd service and not using `ldconfig` is an option, to update the links to the shared libraries, you can use the `LD_LIBRARY_PATH` environment variable. Follow these steps to make sure that Elastic Agent and its spawned processes respect the `LD_LIBRARY_PATH` environment variable.

:::{note}
Make sure that you have administrative privileges to modify the Elastic Agent systemd service configuration.
:::

Steps:
1. Check the status of the Elastic Agent systemd service by running the following command:
`systemctl status elastic-agent.service`
Take note of the path to the elastic-agent.service file, which is typically located in the systemd service directory. Example path: `/etc/systemd/system/elastic-agent.service`
Take note of the path to the `elastic-agent.service` file, which is typically located in the systemd service directory. Example path: `/etc/systemd/system/elastic-agent.service`

2. Open the elastic-agent.service file in your preferred text editor, find the `EnvironmentFile` key (commonly found at `/etc/sysconfig/elastic-agent`), and verify its contents, as these configurations are essential for the elastic-agent's runtime environment initialization. If the EnvironmentFile is absent, create it and set the necessary permissions to ensure the elastic-agent has full access.
2. Open the `elastic-agent.service` file in your preferred text editor, find the `EnvironmentFile` key (commonly found at `/etc/sysconfig/elastic-agent`), and verify its contents, as these configurations are essential for the elastic-agent's runtime environment initialization. If the EnvironmentFile is absent, create it and set the necessary permissions to ensure the elastic-agent has full access.

3. Add the LD_LIBRARY_PATH environment variable to the configured `EnvironmentFile`. You can set it to the directory where libraries (`libclntsh.so`) are located. For example, if your libraries are in the `/opt/oracle/instantclient_21_1 directory`, add the following line to the `EnvironmentFile` (i.e. `/etc/systemd/system/elastic-agent.service`)
3. Add the `LD_LIBRARY_PATH` environment variable to the configured `EnvironmentFile`. Set it to the directory where libraries (`libclntsh.so`) are located. For example, if your libraries are in the `/opt/oracle/instantclient_21_1 directory`, add the following line to the `EnvironmentFile` (i.e. `/etc/systemd/system/elastic-agent.service`)

`LD_LIBRARY_PATH=/opt/oracle/instantclient_21_1`

Expand All @@ -45,22 +71,23 @@ Steps:

`systemctl restart elastic-agent.service`

> Note: Ensure that you replace `/opt/oracle/instantclient_21_1` with the actual path to the directory where the required libraries (`libclntsh.so`) are located. This will set the library search path for the Elastic Agent service to include the specified directory, allowing it to locate the required libraries.
:::{note}
Make sure that you replace `/opt/oracle/instantclient_21_1` with the actual path to the directory where the required libraries (`libclntsh.so`) are located. This will set the library search path for the Elastic Agent service to include the specified directory, allowing it to locate the required libraries.
:::

#### Enable Listener

The Oracle listener is a service that runs on the database host and receives requests from Oracle clients. Make sure that [Listener](https://docs.oracle.com/cd/B19306_01/network.102/b14213/lsnrctl.htm) is be running.
To check if the listener is running or not, run:
The Oracle listener is a service that runs on the database host and receives requests from Oracle clients. Make sure that [Listener](https://docs.oracle.com/cd/B19306_01/network.102/b14213/lsnrctl.htm) is running by using this command:

`lsnrctl STATUS`

If the listener is not running, use the command to start:

`lsnrctl START`

Then, Metricbeat can be launched.
Then, you can launch Metricbeat.

### Oracle DSN Configuration
### Configure Oracle DSN

The following two configuration formats are supported:
```
Expand All @@ -74,25 +101,31 @@ oracle://sys:Oradoc_db1@0.0.0.0:1521/ORCLCDB.localdomain?sysdba=1
user="sys" password="Oradoc_db1" connectString="0.0.0.0:1521/ORCLCDB.localdomain" sysdba=true
```

In the first, URL-based format, special characters should be URL encoded.
In the URL-based format, special characters should be URL encoded.

In the seoncd, logfmt-encoded DSN format, if the password contains a backslash
In the logfmt-encoded DSN format, if the password contains a backslash
character (`\`), it must be escaped with another backslash. For example, if the
password is `my\_password`, it must be written as `my\\_password`.

> Note: To mask the password shown in the DSN, remove the username and password from the DSN string, and configure the DSN to only include the host address and any additional parameters required for the connection. Subsquently, use the `username` and `password` fields under advanced options to configure them.
:::{note}
To mask the password shown in the DSN, remove the username and password from the DSN string, and configure the DSN to only include the host address and any additional parameters required for the connection. Subsquently, use the `username` and `password` fields under advanced options to configure them.
:::

## Compatibility
### Configure Oracle integration to connect to database using TLS Port

This integration has been tested with Oracle Database 19c, and should work for 18c as well though it has not been tested.
To configure Oracle Integration to connect to databases using TLS ports, configure the connection string using this format:

```
user="elastic_ro" password="" connectString="(DESCRIPTION = (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCPS)(HOST=0.0.0.0)(PORT=1832))) (CONNECT_DATA = (SERVICE_NAME = THESERVICENAME)))" sysdba=false
```

### Audit Log

The `database_audit` dataset collects Oracle Audit logs.

**ECS Field Reference**

Please refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.
Refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.

{{fields "database_audit"}}

Expand All @@ -114,7 +147,7 @@ To collect the Tablespace metrics, Oracle integration relies on a specific set o

**ECS Field Reference**

Please refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.
Refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.

{{fields "tablespace"}}

Expand All @@ -130,7 +163,7 @@ To collect the Sysmetrics metrics, Oracle integration relies on a specific set o

**ECS Field Reference**

Please refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.
Refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.

{{fields "sysmetric"}}

Expand All @@ -147,7 +180,7 @@ To collect the Memory metrics, Oracle integration relies on a specific set of vi

**ECS Field Reference**

Please refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.
Refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.

{{fields "memory"}}

Expand All @@ -163,7 +196,7 @@ To collect the System Statistics metrics, Oracle integration relies on a specifi

**ECS Field Reference**

Please refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.
Refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.

{{fields "system_statistics"}}

Expand All @@ -185,7 +218,7 @@ To collect the Performance metrics, Oracle integration relies on a specific set

**ECS Field Reference**

Please refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.
Refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.

{{fields "performance"}}

Expand Down
5 changes: 5 additions & 0 deletions packages/oracle/changelog.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
# newer versions go on top
- version: "1.31.1"
changes:
- description: Update the Oracle Integration documentation.
type: enhancement
link: https://github.com/elastic/integrations/pull/15839
- version: "1.31.0"
changes:
- description: Allow @custom pipeline access to event.original without setting preserve_original_event.
Expand Down
Loading