Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@ _repo.*/
.vscode

.openpublishing.buildcore.ps1
**/.DS_Store
80 changes: 80 additions & 0 deletions powerquery-docs/Connectors/DatabricksCloud.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
---
title: Databricks connector
description: Provides basic information and prerequisites for using Power Query's Databricks connectors.
author: George Chow

ms.topic: conceptual
ms.date: 6/29/2022
ms.author: George Chow
LocalizationGroup: reference
---

# Databricks Cloud

## Summary

| Item | Description |
| ---- | ----------- |
| Release State | Azure Databricks: General Availability<br/>Databricks: Preview |
| Products | Power BI (Datasets)<br/>Power BI (Dataflows) |
| Authentication Types Supported | Azure Active Directory<br/>Organizational Account<br/>Personal Access Token |
| | |

>[!Note]
>Some capabilities may be present in one product but not others due to deployment schedules and host-specific capabilities.


## Prerequisites

You requires either an Azure Databricks account or a paid Databricks Cloud account to use this connector. Databricks Community Edition is not supported.

## Capabilities supported

* Import
* DirectQuery (PowerBI only)

## Connect to Azure Databricks or Databricks Cloud from Power Query Desktop

To connect from Power BI Desktop, take the following steps:

1. In the Get Data experience, search for **databricks** to shortlist the two Databrick connectors: "Azure Databricks" and "Databricks". Choose "Azure Databricks" if your Databricks runs in the Azure cloud; for AWS or GCP, choose "Databricks". (If the entry for "Databricks" is marked with a "(Beta)" suffix, contact your Databricks representative for guidance on using the connector.)

![Get Data from Databricks Cloud.](./media/databricks-cloud/get-data-dbc.png)

2. You will need to provide the "Server hostname" and "HTTP Path" for your Databricks SQL Warehouse. Refer [here](https://docs.microsoft.com/en-us/azure/databricks/integrations/bi/jdbc-odbc-bi#get-server-hostname-port-http-path-and-jdbc-url) for instructions to look up your "Server hostname" and "HTTP Path". Enter these accordingly. You may optionally supply a default catalog and/or database under **Advanced options**. Select **OK** to continue.

![Specify your Databricks SQL Warehouse.](./media/databricks-cloud/azdbc-sql-endpoint.png)

3. You will need provide your credentials to authenticate with your Databricks SQL Warehouse. For Azure Databricks, you can choose between 1) basic user/password; 2) Personal Access Token; and 3) Azure Active Directory (AAD). For AWS or GCP, your choice is between 1) basic user/password; 2) Personal Access Token; and 3) OAuth. Refer [here](https://docs.microsoft.com/en-us/azure/databricks/sql/user/security/personal-access-tokens) for instructions on generating a Personal Access Token (PAT). For organizational account (when using Azure Databricks), you will be prompted to log into your Azure Databricks cluster. For OAuth (when using Databricks on AWS or GCP), you will be prompted to log into your Databricks SQL Warehouse.

![Specify an appropriate token.](./media/databricks-cloud/azdbc-pat.png)

> [!NOTE]
> Once you enter your credentials for a particular **Databricks** cluster, Power BI Desktop caches and reuses those same credentials in subsequent connection attempts. You can modify those credentials by going to **File > Options and settings > Data source settings**. More information: [Change the authentication method](../ConnectorAuthentication.md#change-the-authentication-method)

4. Once you successfully connect, the **Navigator** window shows the data available to you on the cluster. You can choose to either **Transform Data** to transform the data using Power Query or **Load** to load the data in Power Query Desktop.

![Image of Power Query navigator loading Databricks Cloud data to desktop app.](./media/databricks-cloud/navigator-with-filter.jpeg)

## Connect to Databricks Cloud data from Power Query Online

To connect to Databricks Cloud from Power Query Online, take the following steps:

1. In the Get Data experience, select the **Dataflow** category. (Refer [here](https://docs.microsoft.com/en-us/power-bi/transform-model/dataflows/dataflows-create) for instructions.)

![Image of "Start creating your dataflow" dialog box.](./media/databricks-cloud/dataflow-prompt.jpeg)

2. Add the following fragment to the end of your browser URL: ``?pqo_enableDynamicResourcePath=1`` and reload the page. Now navigate to the available data sources and locate the Databrick connectors. You can also use the Search box to shortlist them. Choose "Azure Databricks" if your Databricks runs in the Azure cloud; for AWS or GCP, choose "Databricks".

![Image of the Databricks connectors.](./media/databricks-cloud/filtered-connectors.jpeg)

3. Enter the "Server hostname" and "HTTP Path" for your Databricks cluster. Refer [here](https://docs.microsoft.com/en-us/azure/databricks/integrations/bi/jdbc-odbc-bi#get-server-hostname-port-http-path-and-jdbc-url) for instructions to look up your "Server hostname" and "HTTP Path". You may optionally supply a default catalog and/or database under **Advanced options**.

4. Enter the appropriate credentials. For Azure Databricks, you can choose between 1) basic user/password; 2) Personal Access Token (Account key); and 3) Azure Active Directory (AAD). For AWS or GCP, your choice is between 1) basic user/password; 2) Personal Access Token (Account key); and 3) OAuth (Organizational account).

![Connection Settings and Credentials.](./media/databricks-cloud/azconnect-setting-cred.png)

5. Once you successfully connect, a **Navigator** window appears and displays the data available on the server. Select your data in the navigator. Then select **Next** to transform the data in Power Query.

![Image of Power Query navigator loading Databricks Cloud data to online app.](./media/databricks-cloud/pq-choose-data.png)

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.