Skip to content

Commit

Permalink
Replace deprecated stage name in pre-commit conf (#4209)
Browse files Browse the repository at this point in the history
* Replace deprecated stage name in `pre-commit` conf

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>

* Fix the broken anchor for Databricks job schedules

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>

---------

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>
  • Loading branch information
deepyaman authored Oct 7, 2024
1 parent f1d3751 commit 5491a28
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 2 deletions.
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks

default_stages: [commit, manual]
default_stages: [pre-commit, manual]

repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Here are some typical use cases for running a packaged Kedro project as a Databr

- **Data engineering pipeline**: the output of your Kedro project is a file or set of files containing cleaned and processed data.
- **Machine learning with MLflow**: your Kedro project runs an ML model; metrics about your experiments are tracked in MLflow.
- **Automated and scheduled runs**: your Kedro project should be [run on Databricks automatically](https://docs.databricks.com/workflows/jobs/schedule-jobs.html#add-a-job-schedule).
- **Automated and scheduled runs**: your Kedro project should be [run on Databricks automatically](https://docs.databricks.com/en/jobs/scheduled.html#add-a-job-schedule).
- **CI/CD integration**: you have a CI/CD pipeline that produces a packaged Kedro project.

Running your packaged project as a Databricks job is very different from running it from a Databricks notebook. The Databricks job cluster has to be provisioned and started for each run, which is significantly slower than running it as a notebook on a cluster that has already been started. In addition, there is no way to change your project's code once it has been packaged. Instead, you must change your code, create a new package, and then upload it to Databricks again.
Expand Down

0 comments on commit 5491a28

Please sign in to comment.