Skip to content

update databricks sql warehouse naming #127

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 21, 2023
Merged

update databricks sql warehouse naming #127

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions _posts/2022-06-28-databricks-serverless.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,22 +11,22 @@ author: hamiltonh

We recently migrated Looker to a Databricks SQL Serverless, improving our
infrastructure cost and reducing the footprint of infrastructure we need to
worry about! “Databricks SQL” which provides a single load balanced Endpoint
worry about! “Databricks SQL” which provides a single load balanced Warehouse
for executing Spark SQL queries across multiple Spark clusters behind the
scenes. “Serverless” is an evolution of that concept, rather than running a SQL
Endpoint in our AWS infrastructure, the entirety of execution happens on the
Warehouse in our AWS infrastructure, the entirety of execution happens on the
Databricks side. With a much simpler and faster interface, queries executed in
Looker now return results much faster to our users than ever before!

When we originally provisioned our “Databricks SQL” endpoints, we worked
When we originally provisioned our “Databricks SQL” warehouses, we worked
together with our colleagues at Databricks to ensure [the terraform provider
for Databricks](https://github.com/databricks/terraform-provider-databricks) is
ready for production usage, which as of today is Generally Available. That
original foundation in Terraform allowed us to more easily adopt SQL Serverless
once it was made available to us.

```hcl
resource "databricks_sql_endpoint" "endpoint" {
resource "databricks_sql_warehouse" "warehouse" {
name = "Looker Serverless"
# ...
enable_serverless_compute = true
Expand All @@ -44,9 +44,9 @@ place and setup correctly!

The switch to Serverless reduced our infra configuration and management
footprint, which by itself is an improvement. We also noticed a significant
reduction in cold start times for the SQL Serverless Endpoint compared to the
standard SQL Endpoint. The faster start-up times meant we could configure even
lower auto-terminate times on the endpoint, savings us even more on
reduction in cold start times for the SQL Serverless Warehouse compared to the
standard SQL Warehouse. The faster start-up times meant we could configure even
lower auto-terminate times on the warehouse, savings us even more on
unproductive and idle cluster costs.

On the Looker side there really wasn’t any difference in the connection
Expand Down