pipecutter provides a few tools for luigi such that it works better with data science libraries and environments such as pandas, scikit-learn, and Jupyter notebooks.
-
Updated
Aug 5, 2022 - Python
pipecutter provides a few tools for luigi such that it works better with data science libraries and environments such as pandas, scikit-learn, and Jupyter notebooks.
Universal Luigi ETL pipeline. Validates data received from external sources. Extracts, transforms them and lands.
This repository is based on scraping data from a static website through Luigi. This was created to display my ability to utilize the Luigi pipeline to automatically collect data and other tasks.
Pycon 2017- Creating ETL tasks with Luigi package
Sebagai seorang Data Engineer di Erdigma, tugas ini bertujuan untuk menggabungkan beberapa file CSV yang berasal dari Google Drive ke dalam satu file agar dapat diinput ke database. Pengolahan data ini dilakukan dengan menggunakan Python dan Luigi sebagai workflow automation.
Add a description, image, and links to the luigi-tasks topic page so that developers can more easily learn about it.
To associate your repository with the luigi-tasks topic, visit your repo's landing page and select "manage topics."