Skip to content

Commit 77a6618

Browse files
pyprogrammerblogpyprogrammerblog
authored
Release 0.1.0 (#2)
1 parent ac3ed88 commit 77a6618

39 files changed

+1776
-301
lines changed

‎.github/workflows/main.yml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -26,10 +26,10 @@ jobs:
2626
run: docker-compose run --rm app poetry run pytest
2727

2828
- name: Flake8
29-
run: docker-compose run --rm app poetry run flake8
29+
run: docker-compose run --rm app poetry run flake8 updater
3030

31-
- name: MyPy
32-
run: docker-compose run --rm app poetry run mypy arend --no-strict-optional --ignore-missing-imports
31+
# - name: MyPy
32+
# run: docker-compose run --rm app poetry run mypy updater --no-strict-optional --ignore-missing-imports
3333

3434
- name: Remove Services
3535
run: docker-compose down

‎.pre-commit-config.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,5 +21,5 @@ repos:
2121
rev: v0.971
2222
hooks:
2323
- id: mypy
24-
files: ^tiny_blocks/
24+
files: ^updater/
2525
args: [--no-strict-optional, --ignore-missing-imports]

‎CHANGELOG.md

Lines changed: 1 addition & 102 deletions
Original file line numberDiff line numberDiff line change
@@ -1,109 +1,8 @@
11
Changelog
22
===================================
33

4-
0.1.15 (2022-08-23)
5-
-------------------
6-
7-
- Improve documentation.
8-
- Added examples in notebooks.
9-
10-
11-
0.1.14 (2022-08-23)
12-
-------------------
13-
14-
- Improve documentation.
15-
- Added examples in notebooks.
16-
17-
18-
0.1.13 (2022-08-23)
19-
-------------------
20-
21-
- Improve documentation.
22-
- Added examples in notebooks.
23-
24-
25-
0.1.12 (2022-08-23)
26-
-------------------
27-
28-
- Improve documentation.
29-
30-
31-
0.1.11 (2022-08-22)
32-
-------------------
33-
34-
- Fix in typing for Tee class.
35-
- Improve documentation.
36-
37-
38-
0.1.10 (2022-08-22)
39-
-------------------
40-
41-
- Fix in typing for Tee class.
42-
- Improve documentation.
43-
44-
45-
0.1.9 (2022-08-22)
46-
-------------------
47-
48-
- Fix in typing for Tee class.
49-
- Improve documentation.
50-
51-
52-
0.1.8 (2022-08-21)
53-
-------------------
54-
55-
- Implement Tee.
56-
- Improve documentation.
57-
58-
59-
0.1.7 (2022-08-20)
60-
-------------------
61-
62-
- Implement FanOut.
63-
- Improve documentation.
64-
65-
66-
0.1.6 (2022-08-19)
67-
-------------------
68-
69-
- Improve documentation.
70-
71-
72-
0.1.5 (2022-08-19)
73-
-------------------
74-
75-
- Improve documentation.
76-
77-
78-
0.1.4 (2022-08-19)
79-
-------------------
80-
81-
- Improve documentation.
82-
83-
84-
0.1.3 (2022-08-19)
85-
-------------------
86-
87-
- Improve documentation.
88-
89-
90-
0.1.2 (2022-08-19)
91-
-------------------
92-
93-
- Improve documentation.
94-
- Added new blocks.
95-
- Bug fixes.
96-
97-
98-
0.1.1 (2022-08-16)
99-
-------------------
100-
101-
- Improve documentation.
102-
- Added new blocks.
103-
- Bug fixes.
104-
1054

106-
0.1.0 (2022-08-15)
5+
0.1.0 (2022-09-11)
1076
-------------------
1087

1098
- Initial project structure.

‎README.md

Lines changed: 61 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,36 +1,83 @@
1-
tiny-blocks
2-
=============
1+
progress-updater
2+
=================
33

4-
[![Documentation Status](https://readthedocs.org/projects/tiny-blocks/badge/?version=latest)](https://tiny-blocks.readthedocs.io/en/latest/?badge=latest)
5-
[![License-MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/pyprogrammerblog/tiny-blocks/blob/master/LICENSE)
6-
[![GitHub Actions](https://github.com/pyprogrammerblog/tiny-blocks/workflows/CI/badge.svg/)](https://github.com/pyprogrammerblog/tiny-blocks/workflows/CI/badge.svg/)
7-
[![PyPI version](https://badge.fury.io/py/tiny-blocks.svg)](https://badge.fury.io/py/tiny-blocks)
4+
[![Documentation Status](https://readthedocs.org/projects/progress-updater/badge/?version=latest)](https://progress-updater.readthedocs.io/en/latest/?badge=latest)
5+
[![License-MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/pyprogrammerblog/progress-updater/blob/master/LICENSE)
6+
[![GitHub Actions](https://github.com/pyprogrammerblog/progress-updater/workflows/CI/badge.svg/)](https://github.com/pyprogrammerblog/progress-updater/workflows/CI/badge.svg/)
7+
[![PyPI version](https://badge.fury.io/py/progress-updater.svg)](https://badge.fury.io/py/progress-updater)
88

9-
Tiny Blocks to build large and complex ETL data pipelines!
9+
Writing the progress task to a backend!
1010

1111
Installation
1212
-------------
1313

1414
Install it using ``pip``
1515

1616
```shell
17-
pip install task-updater
17+
pip install progress-updater
1818
```
1919

2020
Basic usage
21-
---------------
21+
-------------
2222

2323
```python
24+
from updater import ProgressUpdater
25+
26+
# create an updater object
27+
updater = ProgressUpdater(task_name="My Task")
28+
29+
with updater(block_name="First part") as updater:
30+
# doing things
31+
updater.notify("doing first block...")
32+
# doing more things
33+
34+
with updater(block_name="Second part"):
35+
# doing things
36+
updater.notify("doing second block...")
37+
# doing more things
38+
39+
updater.raise_latest_exception()
2440
```
2541

26-
Examples
27-
----------------------
42+
Backends
43+
----------
44+
There are three backends available to save our logs.
45+
46+
1. Mongo. See [documentation]().
47+
2. Redis. See [documentation]().
48+
3. SQL. See [documentation]().
49+
50+
51+
Settings
52+
----------
2853

29-
For more complex examples please visit
30-
the [notebooks' folder](https://github.com/pyprogrammerblog/tiny-blocks/blob/master/notebooks/Examples.ipynb).
54+
There are some possible ways to pass settings to the updater.
55+
This is the priority.
56+
57+
1. Passing settings as parameters when creating a `ProgressUpdater` object.
58+
```python
59+
from updater import ProgressUpdater
60+
from updater.backends.mongo import MongoSettings
61+
62+
settings = MongoSettings(
63+
mongo_connection="mongodb://user:pass@mongo:27017",
64+
mongo_db="db",
65+
mongo_collection="logs",
66+
)
67+
68+
with ProgressUpdater(task_name="My Task", settings=settings) as updater:
69+
...
70+
```
71+
72+
2. Environment variables.
73+
The `PU__` prefix indicates that it belongs to `ProgressUpdater`.
74+
```shell
75+
export PU__SQL_DSN=postgresql+psycopg2://user:pass@postgres:5432/db
76+
export PU__SQL_TABLE=logs
77+
```
3178

3279

3380
Documentation
3481
--------------
3582

36-
Please visit this [link](https://tiny-blocks.readthedocs.io/en/latest/) for documentation.
83+
Please visit this [link](https://progress-updater.readthedocs.io/en/latest/) for documentation.

‎README_USERS.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
1-
task-updater
1+
progress-updater
22
=============
33

44
Clone the project and move inside:
55
```shell
6-
git clone https://github.com/pyprogrammerblog/task-updater.git
7-
cd task-updater
6+
git clone https://github.com/pyprogrammerblog/progress-updater.git
7+
cd progress-updater
88
```
99

1010
Install the virtualenv on the root project folder:

‎docker-compose.yaml

Lines changed: 27 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,37 @@
11
version: "3.3"
22
services:
33

4+
redis:
5+
image: redis:4.0.9-alpine
6+
command: redis-server --requirepass 'pass'
7+
ports:
8+
- "6379:6379"
9+
10+
mongo:
11+
image: mongo:latest
12+
environment:
13+
- MONGO_INITDB_ROOT_USERNAME=user
14+
- MONGO_INITDB_ROOT_PASSWORD=pass
15+
ports:
16+
- "27017:27017"
17+
18+
postgres:
19+
image: postgres:latest
20+
environment:
21+
POSTGRES_DB: db
22+
POSTGRES_USER: user
23+
POSTGRES_PASSWORD: pass
24+
ports:
25+
- "5432:5432"
26+
427
app:
528
build:
629
context: .
730
dockerfile: Dockerfile
8-
container_name: 'app'
31+
depends_on:
32+
- redis
33+
- postgres
34+
- mongo
935
environment:
1036
- DOCKER=True
1137
- PYTHONUNBUFFERED=1

‎docs/Makefile

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Minimal makefile for Sphinx documentation
2+
#
3+
4+
# You can set these variables from the command line, and also
5+
# from the environment for the first two.
6+
SPHINXOPTS ?=
7+
SPHINXBUILD ?= sphinx-build
8+
SOURCEDIR = .
9+
BUILDDIR = _build
10+
11+
# Put it first so that "make" without argument is like "make help".
12+
help:
13+
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
14+
15+
.PHONY: help Makefile
16+
17+
# Catch-all target: route all unknown targets to Sphinx using the new
18+
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
19+
%: Makefile
20+
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

‎docs/backends.rst

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
.. _backends:
2+
3+
Backends
4+
=====================
5+
6+
7+
:mod:`Settings`
8+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
9+
10+
.. automodule:: updater.backends
11+
:members: Settings
12+
:exclude-members: Config, backend
13+
14+
15+
:mod:`Mongo`
16+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
17+
18+
.. automodule:: updater.backends.mongo
19+
:members: MongoLog, MongoSettings
20+
:exclude-members: mongo_collection, backend
21+
22+
23+
:mod:`Redis`
24+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
25+
26+
.. automodule:: updater.backends.redis
27+
:members: RedisLog, RedisSettings
28+
:exclude-members: redis_connection, backend
29+
30+
31+
:mod:`SQL`
32+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
33+
34+
.. automodule:: updater.backends.sql
35+
:members: SQLLog, SQLSettings
36+
:exclude-members: sql_session, backend
37+
38+
39+
:mod:`Base Log`
40+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
41+
42+
.. automodule:: updater.backends.log
43+
:members: Log, Logs
44+
:exclude-members:

0 commit comments

Comments
 (0)