pipelines.md

subcategory
Compute

databricks_pipelines Data Source

-> Note If you have a fully automated setup with workspaces created by databricks_mws_workspaces or azurerm_databricks_workspace, please make sure to add depends_on attribute in order to prevent authentication is not configured for provider errors.

Retrieves a list of all databricks_pipeline (Delta Live Tables) ids deployed in a workspace, or those matching the provided search term. Maximum 100 results.

Example Usage

Get all Delta Live Tables pipelines:

data "databricks_pipelines" "all" {}

output "all_pipelines" {
  value = data.databricks_pipelines.all.ids
}

Filter Delta Live Tables pipelines by name (exact match):

data "databricks_pipelines" "this" {
  pipeline_name = "my_pipeline"
}

output "my_pipeline" {
  value = data.databricks_pipelines.this.ids
}

Filter Delta Live Tables pipelines by name (wildcard search):

data "databricks_pipelines" "this" {
  pipeline_name = "%pipeline%"
}

output "wildcard_pipelines" {
  value = data.databricks_pipelines.this.ids
}

Argument Reference

This data source exports the following attributes:

• pipeline_name - (Optional) Filter Delta Live Tables pipelines by name for a given search term. % is the supported wildcard operator.

Attribute Reference

This data source exports the following attributes:

• ids - List of ids for Delta Live Tables pipelines matching the provided search criteria.

Related Resources

The following resources are used in the same context:

• End to end workspace management guide.
• databricks_pipeline to deploy Delta Live Tables.
• databricks_cluster to create Databricks Clusters.
• databricks_job to manage Databricks Jobs to run non-interactive code in a databricks_cluster.
• databricks_notebook to manage Databricks Notebooks.