Skip to content

Commit

Permalink
fix azure storage and tests
Browse files Browse the repository at this point in the history
  • Loading branch information
Alessio Civitillo committed May 8, 2021
1 parent 773d36a commit b78cc37
Show file tree
Hide file tree
Showing 10 changed files with 61 additions and 2,028 deletions.
48 changes: 48 additions & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# Viadot
A simple data ingestion library to guide data flows from some places to other places

## Getting Data from a Source

viadot supports few sources. For instance, the UK Carbon Intensity API does not require credentials.

```python
from viadot.sources.uk_carbon_intensity import UKCarbonIntensity
ukci = UKCarbonIntensity()
ukci.query("/intensity")
ukci.to_df()
```

The above code pulls the UK Carbon Insentity data from the external API to the local Pandas dataframe (df).

## Loading Data to a Source

TODO

## Running tests
```
run.sh
docker exec -it viadot_testing bash
cd tests/ && pytest .
```

## Running flows locally
```
run.sh
poetry shell
FLOW_NAME=supermetrics_to_azure_sql; python -m viadot.flows.$FLOW_NAME
```

## Uploading pkg to PyPi

Generate the `requirements.txt` file from poetry.

```bash
poetry export -f requirements.txt --output requirements.txt --with-credentials --dev
```

And then publish with poetry.

```bash
poetry update
poetry publish --build
```
Loading

0 comments on commit b78cc37

Please sign in to comment.