Anomaly Detection

Want to quickly check if your data is anomalous? Panda Patrol comes pre-built with a basic anomaly detection model using pyod. Just pass the expected values and the actual ones. Panda Patrol will automatically detect anomalies and display them in the UI.

Usage

To generate anomaly detection, you'll need to use the basic_anomaly_detection method. This method takes the following parameters:

• group_name: str - Name of the patrol group.
• expected_data - Input samples with numpy array of shape (n_samples, n_features)
• actual_data - The training input samples with numpy array of shape (n_samples, n_features)
• patrol_name: str (Default: "Basic Anomaly Detection") - The name of the patrol.
• dbt_model_uri=None: str (Default: None. For DBT use only) - The dbt model uri of the Python model that is being tested. For example models/base/first_model.py would be base/first_model. This is used to link the data tests to the dbt model so that Python code related to a DBT test is stored. If excluded, then the data test code will not be stored in the database.

Then you can call the basic_anomaly_detection method anywhere in your data pipeline. For example, you can execute it in a dagster pipeline.

# Import the basic_anomaly_detection method
from panda_patrol.anomaly_detection import basic_anomaly_detection
...
# Run anomaly detection with the expected and actual data
basic_anomaly_detection(
    "numeric tests",
    np.array([2, 4, 7, 9, 15, 16, 23, -5, -80]).reshape(-1, 1),
    np.array([2, 4, 7, 9, 15, 16, 2300, -5, -8000]).reshape(-1, 1),
)

You will then be able to see the results in the UI.