Skip to content

Docker profiles, container omission, fixed job cancellation #43

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 18 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ jobs:

- name: run docker compose
run: |
docker compose up -d --build
docker compose --profile pulsar up -d --build
- name: run docker compose dts
run: |
Expand Down
22 changes: 20 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,23 @@ standard and distributing `TES` tasks execution to `Pulsar` applications.

### Deploy
The most straightforward way to deploy the TESP is to use Docker Compose.

#### API and DB services (default):
```
docker compose up -d
```
Expecting exetrnal Pulsar configured in `settings.toml` before the compose is run.
So far only REST Pulsar communication is supported.

#### With pulsar_rest service:
```
docker compose up -d --build
docker compose --profile pulsar up -d
```

<br />
<br />
<br />

Depending on you Docker and Docker Compose installation, you may need to use `docker-compose` (with hyphen) instead.

You might encounter a timeout error in container runtime which can be solved by correct `mtu` configuration either in the `docker-compose.yaml`:
Expand Down Expand Up @@ -180,7 +194,11 @@ Service representing `TESP API` is configured to mount this project sources as a
same command as is mentioned above. Therefore, any changes made to the sources in this repository will be immediately applied to the docker
service as well, enabling live reloading which makes development within the `docker` environment very easy.
```shell
docker-compose up -d
docker compose up -d
```
Or
```shell
docker compose --profile pulsar up -d
```

&nbsp;
Expand Down
4 changes: 1 addition & 3 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -39,15 +39,13 @@ services:
dockerfile: Dockerfile
target: development
image: pulsar_rest
profiles: ["pulsar"]
container_name: pulsar-rest
privileged: true
expose:
- "8913"
volumes:
- ./docker/pulsar_rest/app.yml:/opt/pysetup/app.yml
# DIND - problem with resource limitation, missing cgroups inside
# - ./docker/pulsar_rest/data/:/opt/pysetup/files/staging/
# DOOD
- /opt/pysetup/files/staging/:/opt/pysetup/files/staging/
- /var/run/docker.sock:/var/run/docker.sock

Expand Down
2 changes: 1 addition & 1 deletion docker/tesp_api/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# See https://docs.docker.com/develop/develop-images/multistage-build/

# Environment variables and base image
FROM python:3.10-slim-buster as python-base
FROM python:3.10-slim-bullseye as python-base
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
PIP_NO_CACHE_DIR=off \
Expand Down
9 changes: 6 additions & 3 deletions tesp_api/api/endpoints/task_endpoints.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
from pymonad.promise import Promise
from fastapi.params import Depends
from fastapi import APIRouter, Body
from fastapi.responses import Response
from fastapi.responses import Response, JSONResponse

from tesp_api.api.error import api_handle_error
from tesp_api.service.event_dispatcher import dispatch_event
Expand Down Expand Up @@ -111,15 +111,18 @@ async def cancel_task(
maybe_of(token_subject),
ObjectId(id)
))).then(lambda get_tasks_args: task_repository.cancel_task(*get_tasks_args))\
.map(lambda task_id: Response(status_code=200, media_type="application/json"))\
.map(lambda task_id_maybe:
# Use JSONResponse to ensure the body is "{}"
JSONResponse(content={}, status_code=200)
)\
.catch(api_handle_error)


@router.get("/service-info",
responses={200: {"description": "Ok"}},
description=descriptions["service-info"],
response_model=TesServiceInfo)
async def get_service_info() -> TesServiceInfo:
async def get_service_info() -> TesServiceInfo: # FastAPI directly handles Pydantic model return
return TesServiceInfo(
id="fi.muni.cz.tesp",
name="TESP",
Expand Down
Loading
Loading