Skip to content

🎉 Migrate from fastapi-cloud-tasks to fastapi-gcp-tasks #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
May 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions .flake8

This file was deleted.

48 changes: 48 additions & 0 deletions .github/workflows/lint.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
name: lint

on:
push:
branches:
- master
pull_request:
types:
- opened
- synchronize

jobs:
lint:
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@v4
- name: Set up python
id: setup-python
uses: actions/setup-python@v5
with:
python-version: '3.11'

- name: Install Poetry
uses: snok/install-poetry@v1
with:
virtualenvs-create: true
virtualenvs-in-project: true
installer-parallel: true

- name: Load cached venv
id: cached-poetry-dependencies
uses: actions/cache@v3
with:
path: .venv
key: venv-${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-${{ hashFiles('**/poetry.lock') }}

- name: Install dependencies
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
run: poetry install --no-interaction --no-root

- name: Install project
run: poetry install --no-interaction

- name: Run tests
run: |
source .venv/bin/activate
sh scripts/lint.sh
6 changes: 5 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -132,4 +132,8 @@ dmypy.json
pyvenv.cfg
bin/

version.txt
# repository metadata
version.txt

# JetBrains IDE
.idea/
7 changes: 0 additions & 7 deletions .isort.cfg

This file was deleted.

15 changes: 0 additions & 15 deletions .pre-commit-config.yaml

This file was deleted.

1 change: 1 addition & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
MIT License

Copyright (c) 2024 Simplify Jobs, Inc
Copyright (c) 2021 Adori Labs, Inc

Permission is hereby granted, free of charge, to any person obtaining a copy
Expand Down
38 changes: 27 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,16 +61,20 @@ pip install fastapi-cloud-tasks
### Delayed job

```python
from fastapi_cloud_tasks import DelayedRouteBuilder
from fastapi_gcp_tasks import DelayedRouteBuilder

delayed_router = APIRouter(route_class=DelayedRouteBuilder(...))


class Recipe(BaseModel):
ingredients: List[str]
ingredients: List[str]


@delayed_router.post("/{restaurant}/make_dinner")
async def make_dinner(restaurant: str, recipe: Recipe):
# Do a ton of work here.


# Do a ton of work here.


app.include_router(delayed_router)
Expand All @@ -89,22 +93,28 @@ make_dinner.options(countdown=1800).delay(...)
```

### Scheduled Task

```python
from fastapi_cloud_tasks import ScheduledRouteBuilder
from fastapi_gcp_tasks import ScheduledRouteBuilder

scheduled_router = APIRouter(route_class=ScheduledRouteBuilder(...))


class Recipe(BaseModel):
ingredients: List[str]
ingredients: List[str]


@scheduled_router.post("/home_cook")
async def home_cook(recipe: Recipe):
# Make my own food


# Make my own food

app.include_router(scheduled_router)

# If you want to make your own breakfast every morning at 7AM IST.
home_cook.scheduler(name="test-home-cook-at-7AM-IST", schedule="0 7 * * *", time_zone="Asia/Kolkata").schedule(recipe=Recipe(ingredients=["Milk","Cereal"]))
home_cook.scheduler(name="test-home-cook-at-7AM-IST", schedule="0 7 * * *", time_zone="Asia/Kolkata").schedule(
recipe=Recipe(ingredients=["Milk", "Cereal"]))
```

## Concept
Expand Down Expand Up @@ -238,7 +248,7 @@ Check the fleshed out example at [`examples/full/tasks.py`](examples/full/tasks.

If you're not running on CloudRun and want to an OAuth Token instead, you can use the `oauth_task_hook` instead.

Check [fastapi_cloud_tasks/hooks.py](fastapi_cloud_tasks/hooks.py) to get the hang od hooks and how you can use them.
Check [fastapi_cloud_tasks/hooks.py](fastapi_gcp_tasks/hooks.py) to get the hang od hooks and how you can use them.

## Configuration

Expand Down Expand Up @@ -301,7 +311,7 @@ Usage:
simple_task.options(...).delay()
```

All options from above can be overriden per call (including DelayedRouteBuilder options like `base_url`) with kwargs to the `options` function before calling delay.
All options from above can be overwritten per call (including DelayedRouteBuilder options like `base_url`) with kwargs to the `options` function before calling delay.

Example:

Expand Down Expand Up @@ -355,12 +365,18 @@ async def my_task(ct_headers: CloudTasksHeaders = Depends()):
print(ct_headers.queue_name)
```

Check the file [fastapi_cloud_tasks/dependencies.py](fastapi_cloud_tasks/dependencies.py) for details.
Check the file [fastapi_cloud_tasks/dependencies.py](fastapi_gcp_tasks/dependencies.py) for details.

## Contributing

- Run `pre-commit install` on your local to get pre-commit hook.
- Make changes and raise a PR!
- If the change is massive, open an issue to discuss it before writing code.

Note: This project is neither affiliated with, nor sponsored by Google.
## License

This project is licensed under the terms of the MIT license. This project was forked from [fastapi-cloud-tasks](https://github.com/Adori/fastapi-cloud-tasks) under the MIT license. All changes made to the original project are also licensed under the MIT license.

## Disclaimer

This project is neither affiliated with, nor sponsored by Google.
11 changes: 3 additions & 8 deletions examples/full/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,13 @@
from uuid import uuid4

# Third Party Imports
from fastapi import FastAPI
from fastapi import Response
from fastapi import status
from fastapi import FastAPI, Response, status
from google.api_core.exceptions import AlreadyExists

# Imports from this repository
from examples.full.serializer import Payload
from examples.full.settings import IS_LOCAL
from examples.full.tasks import fail_twice
from examples.full.tasks import hello
from examples.full.tasks import fail_twice, hello

app = FastAPI()

Expand Down Expand Up @@ -44,9 +41,7 @@ async def deduped(response: Response):
@app.get("/fail")
async def fail():
fail_twice.delay()
return {
"message": "The triggered task will fail twice and then be marked done automatically"
}
return {"message": "The triggered task will fail twice and then be marked done automatically"}


# We can use a trick on local to get all tasks on the same process as the main server.
Expand Down
2 changes: 2 additions & 0 deletions examples/full/serializer.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,6 @@


class Payload(BaseModel):
"""Basic payload from the api."""

message: str
14 changes: 4 additions & 10 deletions examples/full/settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,18 @@
import os

# Third Party Imports
from google.cloud import scheduler_v1
from google.cloud import tasks_v2
from google.cloud import scheduler_v1, tasks_v2

# Imports from this repository
from fastapi_cloud_tasks.utils import location_path
from fastapi_cloud_tasks.utils import queue_path
from fastapi_gcp_tasks.utils import location_path, queue_path

# set env var IS_LOCAL=false for your deployment environment
IS_LOCAL = os.getenv("IS_LOCAL", "true").lower() == "true"

# The suffix _fastapi_cloud_tasks is a trick for running both main and task server in the same process for local
# In a deployed environment, you'd most likely want them to be separate
# Check main.py for how this is used.
TASK_LISTENER_BASE_URL = os.getenv(
"TASK_LISTENER_BASE_URL", default="http://localhost:8000/_fastapi_cloud_tasks"
)
TASK_LISTENER_BASE_URL = os.getenv("TASK_LISTENER_BASE_URL", default="http://localhost:8000/_fastapi_cloud_tasks")
TASK_PROJECT_ID = os.getenv("TASK_PROJECT_ID", default="sample-project")
TASK_LOCATION = os.getenv("TASK_LOCATION", default="asia-south1")
SCHEDULED_LOCATION = os.getenv("SCHEDULED_LOCATION", default="us-central1")
Expand All @@ -41,9 +37,7 @@
location=SCHEDULED_LOCATION,
)

TASK_OIDC_TOKEN = tasks_v2.OidcToken(
service_account_email=TASK_SERVICE_ACCOUNT, audience=TASK_LISTENER_BASE_URL
)
TASK_OIDC_TOKEN = tasks_v2.OidcToken(service_account_email=TASK_SERVICE_ACCOUNT, audience=TASK_LISTENER_BASE_URL)
SCHEDULED_OIDC_TOKEN = scheduler_v1.OidcToken(
service_account_email=TASK_SERVICE_ACCOUNT, audience=TASK_LISTENER_BASE_URL
)
40 changes: 21 additions & 19 deletions examples/full/tasks.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,30 +2,32 @@
import logging

# Third Party Imports
from fastapi import Depends
from fastapi import FastAPI
from fastapi import Depends, FastAPI
from fastapi.routing import APIRouter
from google.protobuf import duration_pb2

# Imports from this repository
from examples.full.serializer import Payload
from examples.full.settings import CLOUD_TASKS_EMULATOR_URL
from examples.full.settings import IS_LOCAL
from examples.full.settings import SCHEDULED_LOCATION_PATH
from examples.full.settings import SCHEDULED_OIDC_TOKEN
from examples.full.settings import TASK_LISTENER_BASE_URL
from examples.full.settings import TASK_OIDC_TOKEN
from examples.full.settings import TASK_QUEUE_PATH
from fastapi_cloud_tasks import DelayedRouteBuilder
from fastapi_cloud_tasks.dependencies import CloudTasksHeaders
from fastapi_cloud_tasks.dependencies import max_retries
from fastapi_cloud_tasks.hooks import chained_hook
from fastapi_cloud_tasks.hooks import deadline_delayed_hook
from fastapi_cloud_tasks.hooks import deadline_scheduled_hook
from fastapi_cloud_tasks.hooks import oidc_delayed_hook
from fastapi_cloud_tasks.hooks import oidc_scheduled_hook
from fastapi_cloud_tasks.scheduled_route import ScheduledRouteBuilder
from fastapi_cloud_tasks.utils import emulator_client
from examples.full.settings import (
CLOUD_TASKS_EMULATOR_URL,
IS_LOCAL,
SCHEDULED_LOCATION_PATH,
SCHEDULED_OIDC_TOKEN,
TASK_LISTENER_BASE_URL,
TASK_OIDC_TOKEN,
TASK_QUEUE_PATH,
)
from fastapi_gcp_tasks import DelayedRouteBuilder
from fastapi_gcp_tasks.dependencies import max_retries
from fastapi_gcp_tasks.hooks import (
chained_hook,
deadline_delayed_hook,
deadline_scheduled_hook,
oidc_delayed_hook,
oidc_scheduled_hook,
)
from fastapi_gcp_tasks.scheduled_route import ScheduledRouteBuilder
from fastapi_gcp_tasks.utils import emulator_client

app = FastAPI()

Expand Down
7 changes: 4 additions & 3 deletions examples/simple/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,8 @@
from pydantic.v1 import BaseModel

# Imports from this repository
from fastapi_cloud_tasks import DelayedRouteBuilder
from fastapi_cloud_tasks.utils import emulator_client
from fastapi_cloud_tasks.utils import queue_path
from fastapi_gcp_tasks import DelayedRouteBuilder
from fastapi_gcp_tasks.utils import emulator_client, queue_path

# set env var IS_LOCAL=false for your deployment environment
IS_LOCAL = os.getenv("IS_LOCAL", "true").lower() == "true"
Expand Down Expand Up @@ -39,6 +38,8 @@


class Payload(BaseModel):
"""Basic payload from the api."""

message: str


Expand Down
5 changes: 0 additions & 5 deletions fastapi_cloud_tasks/__init__.py

This file was deleted.

6 changes: 0 additions & 6 deletions fastapi_cloud_tasks/decorators.py

This file was deleted.

15 changes: 0 additions & 15 deletions fastapi_cloud_tasks/exception.py

This file was deleted.

Loading
Loading