Skip to content

Commit

Permalink
feat: add readme in schedulers
Browse files Browse the repository at this point in the history
  • Loading branch information
FlorentLvr committed Apr 11, 2023
1 parent 212d4e5 commit a665736
Showing 1 changed file with 24 additions and 0 deletions.
24 changes: 24 additions & 0 deletions models/advanced/schedulers/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Schedulers

## Description
The schedulers folder is an essential component of your data management system. It stores pipeline files that can be scheduled using the naas.scheduler feature. However, it's important to ensure that the execution time is lower than the cron time. If your pipeline lasts for 15 minutes and you want it to update every 10 minutes, you might encounter data issues. In this case, we recommend building smaller pipelines scheduled more frequently, along with a larger one scheduled 2-3 times a day. Of course, everything depends on your specific requirements.

By utilizing the schedulers folder, you can automate data-related tasks and ensure that your data is always up-to-date. This, in turn, can lead to better business decisions and improved operational efficiency.


## Notebooks

### __pipeline__.ipynb

The **__pipeline__** file is a special file that specifies the order in which the models in the Bronze, Silver, and Gold categories should be trained and evaluated.
This file is used to automate the model selection process, ensuring that the most accurate model is always used for a given task.

## Folder

### pipeline_executions

The pipeline_executions folder store all of the notebooks that are executed during your pipeline.
For each execution, a new folder will be created with a unique timestamp and ID.

This folder is important because it allows you to easily track the progress of your pipeline and identify any issues that may arise. By examining the notebooks in the pipeline_executions folder, you can quickly identify which steps in the pipeline succeeded and which steps failed. This can be extremely helpful when debugging your pipeline or trying to understand why certain results were produced.

0 comments on commit a665736

Please sign in to comment.