README.md

Viadot

---

Documentation: https://dyvenia.github.io/viadot/

Source Code: https://github.com/dyvenia/viadot

---

A simple data ingestion library to guide data flows from some places to other places.

Getting Data from a Source

Viadot supports several API and RDBMS sources, private and public. Currently, we support the UK Carbon Intensity public API and base the examples on it.

from viadot.sources.uk_carbon_intensity import UKCarbonIntensity
ukci = UKCarbonIntensity()
ukci.query("/intensity")
ukci.to_df()

The above code pulls data from the API to a pandas DataFrame.

Loading Data to a Source

TODO

Running tests

run.sh
docker exec -it viadot_testing bash
cd tests/ && pytest .

Running flows locally

run.sh
FLOW_NAME=supermetrics_to_azure_sql; python -m viadot.flows.$FLOW_NAME