Pre-requisites (all users)
- Download and install Docker Desktop via Docker Desktop for Mac or Docker Desktop for Windows. (❗This may require a machine restart if you have not downloaded it before.)
- Clone this repo and open the project in your IDE (I am using Visual Studio Code)
Mac Users
This was tested on a Macbook Pro (M2) running Ventura 13.4.1
- To get started, from the project root create a
.envfile and copy-🍝 the contents of.env.sampleinto it (refer to the "Environment Variables" table below for local values) - Create your virtual environment (for
venvusers, this is:python3 -m venv api && source api/bin/activate) - Still from the root, run
make api-start-localto set up the database, load requirements, and start the api. - At the time of this writing, the database will be created without any stub data to use. It's important to note that, due to the linkage between
PersonandEvent, you will need to create somePersonrows before creating anEvent. This can be done easily if you have Postman installed and upload the collection I included inapi/docs/User Events Api Application.postman_collection.json.
Windows Users
This was tested on a Samsung machine (64-bit) running Windows 11
- To get started, from the project root create a
.envfile and copy-🍝 the contents of.env.sampleinto it (refer to the "Environment Variables" table below for local values) - Create your virtual environment (for
venvusers, this is:python3 -m venv api, thenapi/Scripts/activate) - Still from the root, run
bash startup.shto set up the database, load requirements, and start the api (see "Troubleshooting Setup" inapi/docs/troubleshooting_setup.mdbelow if you receive errors) - At the time of this writing, the database will be created without any stub data to use. It's important to note that, due to the linkage between
PersonandEvent, you will need to create somePersonrows before creating anEvent. This can be done easily if you have Postman installed and upload the collection I included inapi/docs/User Events Api Application.postman_collection.json.
👇 Click the dropdown tables below for additional resources
Environment Variables
While there is only one environment currently (local), variables for database connections, authentication, and other sensitive information would need to be changed for non-local development.
| Env Variable | Local Value | Description & Usage |
|---|---|---|
DB_HOST |
"localhost" |
The host for the PostgreSQL database |
DB_NAME |
"postgres" |
The name of the PostgreSQL database |
DB_PASS |
"password" |
The password for the PostgreSQL database |
DB_PORT |
6543 |
The port for the PostgreSQL database |
DB_USER |
"dev" |
The user of the PostgreSQL database (PostgreSQL requires this) |
ENVIRONMENT |
"local" |
Denotes the current development environment |
SHOW_ERROR_DETAILS |
false |
Allows exception details to be surfaced directly from failing web requests as described here. To avoid security issues, this is false by default and will only be true if ENVIRONMENT="local" |
Tasks & Commands
| Command | Description |
|---|---|
make api-start-local |
Sets up the database in a Docker container, loads requirements, and starts the api locally. This is a composite command consisting of setup-db, load-requirements, and start-api. |
make load-requirements |
Loads requirements from requirements.txt. |
make setup-db |
Sets up a PostgreSQL database in a Docker container. |
make start-api |
Starts the api (assuming the database is already running successfully and requirements are loaded). |
bash startup.sh |
A shell command equivalent of make api-start-local; Intended for Windows users (or anyone not using a Makefile) to reduce annoying installations and setup but it can be used by Mac users as well. |
OpenApi Documentation can be found locally at http://127.0.0.1:8080/docs. Additionally, a Postman Collection has been included for convenience (see api/docs/User Events Api Application.postman_collection.json). Import it directly into Postman to begin playing with the api. It includes collection variables and pre-request scripts for dynamic POST requests, but it does not include environment variables. You will need to add an environment variable yourself called base_url. It's value should be http://127.0.0.1:8080.
For those who don't have Postman, Routes & Requests are included in the dropdowns below
GET Persons
Route: http://127.0.0.1:8080/persons
Response:
{
"data": [
{
"id": "f4cc4f34-3f7c-4a0c-b333-df23f72ebc68",
"datetime_created": "2023-10-08T06:03:33.330037",
"datetime_modified": "2023-10-08T06:03:33.330037",
"email": "Max.Rolfson@test.mock",
"events": [
{
"id": "3b027982-2fe9-4491-a5a2-b504b0e2bb8d",
"person_id": "f4cc4f34-3f7c-4a0c-b333-df23f72ebc68",
"event_type": "signup",
"datetime_created": "2023-10-08 06:03:33.334854"
}
],
"first_name": "Max",
"last_name": "Rolfson",
"role": "user"
},
{
"id": "40a349f1-35d7-48d7-aa09-bb0afdd35e3e",
"datetime_created": "2023-10-07T23:36:07.478145",
"datetime_modified": "2023-10-07T19:24:37.085629",
"email": "Jean.Grey@phoenix.mock",
"events": [
{
"id": "5d114a9c-9dab-4ceb-8c8e-25fe30b4f099",
"person_id": "40a349f1-35d7-48d7-aa09-bb0afdd35e3e",
"event_type": "click",
"datetime_created": "2023-10-07T23:36:54.430671"
},
{
"id": "d8b769c7-aa0b-4ddc-b63e-42fdbaa3981e",
"person_id": "40a349f1-35d7-48d7-aa09-bb0afdd35e3e",
"event_type": "click",
"datetime_created": "2023-10-07T23:36:55.224951"
},
{
"id": "d156191e-f073-45f6-9031-c4b323fa1666",
"person_id": "40a349f1-35d7-48d7-aa09-bb0afdd35e3e",
"event_type": "signup",
"datetime_created": "2023-10-07T23:36:56.114097"
},
{
"id": "3669c226-b6ca-4275-a68a-d447ca15cf88",
"person_id": "40a349f1-35d7-48d7-aa09-bb0afdd35e3e",
"event_type": "signup",
"datetime_created": "2023-10-07T23:36:56.935621"
}
],
"first_name": "Jean",
"last_name": "Grey",
"role": "admin"
},
{
"id": "9f5e7b72-3268-44c0-b5a7-98fed5b8d740",
"datetime_created": "2023-10-08T05:39:36.671925",
"datetime_modified": "2023-10-08T00:57:48.993778",
"email": "scott.summers@themoon.mock",
"events": [
{
"id": "0323d885-13a9-460e-96ab-001afc534fcb",
"person_id": "9f5e7b72-3268-44c0-b5a7-98fed5b8d740",
"event_type": "signup",
"datetime_created": "2023-10-08T05:39:36.683119"
},
{
"id": "7b20a3bd-002b-4d58-941f-c79011ba3e1a",
"person_id": "9f5e7b72-3268-44c0-b5a7-98fed5b8d740",
"event_type": "click",
"datetime_created": "2023-10-08T00:36:54.430671"
},
{
"id": "d7584bb53-bef2-49c2-8490-8780c85d8744",
"person_id": "9f5e7b72-3268-44c0-b5a7-98fed5b8d740",
"event_type": "click",
"datetime_created": "2023-10-08T00:58:55.224951"
}
],
"first_name": "Scott",
"last_name": "Summers",
"role": "admin"
}
],
"response": {
"details": "The request was successful",
"message": "Ok",
"status": 200
}
}
GET Person by id
Route: http://127.0.0.1:8080/persons/{id}
Response:
{
"data": {
"id": "9f5e7b72-3268-44c0-b5a7-98fed5b8d740",
"datetime_created": "2023-10-08T05:39:36.671925",
"datetime_modified": "2023-10-08T00:57:48.993778",
"email": "scott.summers@themoon.mock",
"events": [
{
"id": "0323d885-13a9-460e-96ab-001afc534fcb",
"person_id": "9f5e7b72-3268-44c0-b5a7-98fed5b8d740",
"event_type": "signup",
"datetime_created": "2023-10-08T05:39:36.683119"
},
{
"id": "7b20a3bd-002b-4d58-941f-c79011ba3e1a",
"person_id": "9f5e7b72-3268-44c0-b5a7-98fed5b8d740",
"event_type": "click",
"datetime_created": "2023-10-08T00:36:54.430671"
},
{
"id": "d7584bb53-bef2-49c2-8490-8780c85d8744",
"person_id": "9f5e7b72-3268-44c0-b5a7-98fed5b8d740",
"event_type": "click",
"datetime_created": "2023-10-08T00:58:55.224951"
}
],
"first_name": "Scott",
"last_name": "Summers",
"role": "admin"
},
"response": {
"details": "The request was successful",
"message": "Ok",
"status": 200
}
}
POST Persons
Route: http://127.0.0.1:8080/persons
Request body:
{
"email": "kate.pryde@marauders.mock",
"first_name": "Kate",
"last_name": "Pryde",
"role": 'admin'
}
Response:
{
"data": {
"id": "f4cc4f34-3f7c-4a0c-b333-df23f72ebc68",
"datetime_created": "2023-10-08T06:03:33.330037",
"datetime_modified": "2023-10-08T06:03:33.330037",
"email": "kate.pryde@marauders.mock",
"events": [
{
"id": "3b027982-2fe9-4491-a5a2-b504b0e2bb8d",
"datetime_created": "2023-10-08T06:03:33.334854",
"event_type": "signup",
"person_id": "f4cc4f34-3f7c-4a0c-b333-df23f72ebc68"
}
],
"first_name": "Kate",
"last_name": "Pryde",
"role": "admin"
},
"response": {
"details": "The request was successful",
"message": "Ok",
"status": 201
}
}
DELETE Persons
Route: http://127.0.0.1:8080/persons/{id}
Request body:
{
"id": "7c56a5bc-6037-432f-bd4e-3606a744fcf4"
}
Response:
{
"data": null,
"response": {
"details": "The request was successful",
"message": "Ok",
"status": 200
}
}
GET Events
Route: http://127.0.0.1:8080/events
Params (optional):
?keyword={event_type}: Use a keyword to search events?person_id={person_id}: Use a person_id (person.id) to search events- Example:
http://127.0.0.1:8080/events?keyword=click&person_id=40a349f1-35d7-48d7-aa09-bb0afdd35e3e
Response (no params):
{
"data": [
{
"id": "bf8f9aa1-94b9-4261-9ccb-a35e279d2af2",
"datetime_created": "2023-10-08T06:21:29.214489",
"event_type": "submitted_feedback",
"person_id": "9f5e7b72-3268-44c0-b5a7-98fed5b8d740"
},
{
"id": "cb9ce32a-6571-4e7c-8f6d-ffbdd31aade2",
"datetime_created": "2023-10-08T20:00:37.769065",
"event_type": "signup",
"person_id": "427d6a9e-7d7c-4dc7-a2e5-f38128421ab9"
},
{
"id": "f5006d03-aad9-4b23-aa14-9152de9e05dd",
"datetime_created": "2023-10-08T20:05:49.174802",
"event_type": "signup",
"person_id": "ea880a90-d96f-4d80-8bbe-71436f12cd4f"
},
{
"id": "5d114a9c-9dab-4ceb-8c8e-25fe30b4f099",
"datetime_created": "2023-10-07T23:36:54.430671",
"event_type": "click",
"person_id": "40a349f1-35d7-48d7-aa09-bb0afdd35e3e"
},
{
"id": "d8b769c7-aa0b-4ddc-b63e-42fdbaa3981e",
"datetime_created": "2023-10-07T23:36:55.224951",
"event_type": "click",
"person_id": "40a349f1-35d7-48d7-aa09-bb0afdd35e3e"
},
{
"id": "d156191e-f073-45f6-9031-c4b323fa1666",
"datetime_created": "2023-10-07T23:36:56.114097",
"event_type": "signup",
"person_id": "40a349f1-35d7-48d7-aa09-bb0afdd35e3e"
},
{
"id": "3669c226-b6ca-4275-a68a-d447ca15cf88",
"datetime_created": "2023-10-07T23:36:56.935621",
"event_type": "signup",
"person_id": "40a349f1-35d7-48d7-aa09-bb0afdd35e3e"
},
{
"id": "0323d885-13a9-460e-96ab-001afc534fcb",
"datetime_created": "2023-10-08T05:39:36.683119",
"event_type": "signup",
"person_id": "9f5e7b72-3268-44c0-b5a7-98fed5b8d740"
}
],
"response": {
"details": "The request was successful",
"message": "Ok",
"status": 200
}
}
GET Events by id
Route: http://127.0.0.1:8080/events/{id}
Response:
{
"data": {
"id": "d8b769c7-aa0b-4ddc-b63e-42fdbaa3981e",
"datetime_created": "2023-10-07T23:36:55.224951",
"event_type": "click",
"person_id": "40a349f1-35d7-48d7-aa09-bb0afdd35e3e"
},
"response": {
"details": "The request was successful",
"message": "Ok",
"status": 200
}
}
POST Events
Route: http://127.0.0.1:8080/events/{id}
Request body:
{
"event_type": "signup",
"person_id": "9c65af1a-c109-4c17-9bf1-5f4bcac95e3c"
}
Response:
{
"data": {
"id": "9c65af1a-c109-4c17-9bf1-5f4bcac95e3c",
"datetime_created": "2023-10-06T10:53:31.283195",
"event_type": "signup",
"person_id": "0fcf3634-9b0c-4ceb-ab53-7ba8edf3d5fa"
},
"response": {
"details": "The request was successful",
"message": "Ok",
"status": 201
}
}
DELETE Events
Route: http://127.0.0.1:8080/events/{id}
Request body:
{
"id": "62701308-809d-4302-b345-92ca72285194",
"person_id": "049bb5dd-91d2-464e-b049-07da3e8d2627"
}
Response:
{
"data": null,
"response": {
"details": "The request was successful",
"message": "Ok",
"status": 200
}
}
Migrations are managed with the piccolo package.
New migrations need to be created whenever there is a change to the database schema. To create a new migration, use the following (always include a brief description): piccolo migrations new db --desc="good description of migration here".
To apply the migration, use piccolo migrations forwards all.
Originally I wanted to create an api that allowed for the creation of both events and users. I already had a boilerplate CRUD api that I was working on and it had Person objects so I thought, "Why not?". For this reason, Persons are included in the project. The linkage between Person and Event isn't complete but I'm still working on that. I focused my attention on the Event functionality, which includes the ability to search by a keyword.
I chose blacksheep for my framework because (A) I was already playing with it, (B) it's new and different to me, and (C) I am fascinated by its blend of Python + .NET. It seems like the best of both worlds and I was intrigued by the promise of maintainability, type safety, and speed of performance. I used Piccolo ORM for database interactions because it was recommended by Blacksheep's Roberto Prevato but also because after a galumping through SQLAlchemy docs for what felt like decades, 30 minutes spent on Piccolo's offerings convinced me it was feature-rich and worth trying (the auto_increment option for a Timestamp, the Playground, piccolo admin, and simple Postgres setup are all great examples). For data validation, I am using Pydantic, which handily ships with Piccolo.
As for architecture, I was going for a microservices approach. This is a small project so I began with a single file (server.py) which contains the database connection logic, routing, and 'service layer methods'. I do not enjoy how crowded it has become and I intend to simplify everything by separating routes from service logic and streamlining errors, exceptions, responses and if/else conditions. After doing some light reading about events and event sourcing, I decided to use a very simple Event table structure and a JSONB string which can be searched with the arrow function provided by Piccolo (in the same way as a ->> is used in raw SQL).
[1] Simple CRUD REST API with Blacksheep and Piccolo ORM by Carlos Armando Marcano Vargas
[2] Auto Migrations (from the Piccolo blog)
[3] Piccolo ORM
[4] "How to Decide What Events to Track (+15 Examples)" by Levi Olmstead on Whatfix Blog
[5] "Event Storage in Postgres" by Kasey Speakman
[6] "Building an Event Storage" by Kasey Speakman