Back end

This repository provides a news website server written in Haskell.

Overview

Functional requirements

• Most functional requirements are described here.

OpenAPI 3

• Swagger UI.

Security

Authorization

We provide user authorization via JWT.

• JWT.
• What Are Refresh Tokens and How to Use Them Securely.

Encryption

In a database, we store user passwords encrypted via password.

Infrastructure

• Pulumi
• Kubernetes
• GitHub Actions

Database

• PostgreSQL.
• esqueleto.
  • esqueleto prevents SQL injections.

Architecture

Haskell modules from back/src are logically grouped into the following units.

• Persist - provides operations with a database.
• Service - provides services on top of Persist.
• API - describes API.
• Controller - provides operations for API endpoints on top of Service.
• Server - binds API endpoints and Controller operations to produce a server.

Development

1. Nix prerequisites

   See these for additional info:

   • codium-generic - info just about VSCodium and extensions.
   • codium-haskell - an advanced version of this flake.
   • flake.nix - extensively commented code.
   • Haskell
   • Troubleshooting
   • Prerequisites

2. Install Nix - see how.

3. Install the following programs.

   • pulumi
   • microk8s

4. Start a devShell.

   nix develop

5. (Optionally) Write settings.json and start VSCodium.

   nix run .#writeSettings
   nix run .#codium .

6. (Optionally) Open a .hs file and hover over a function. Shortly, Haskell Language Server should start giving the type info.

7. See the following sections. They assume dev environment.

   • There are configurations for the prod environment.

Run the application

Cluster

Run a database, a server, and tests in a Kubernetes cluster.

1. Create a cluster.

   # prepare the environment
   source microk8s.sh
   
   # start a microk8s vm
   microk8s start

2. The pods configuration is in Pulumi.dev.yaml.

3. Prepare pulumi.

   # go to pulumi directory
   cd pulumi
   
   # install node packages
   npm i
   
   # select dev stack
   pulumi stack select dev

4. Create pods in the cluster.

   # create all resources
   pulumi up
   
   # select `yes`
   
   # check that all pods are ready
   kubectl get po -n breaking-news-dev

Cabal

1. Follow the steps from Cluster until pod creation.

2. Run a database in the cluster.

   # create postgres-related resources
   pulumi up -t **postgres*
   
   # select `yes`
   
   kubectl get po -n breaking-news-dev

3. The app configuration is in the local directory. See back.dev.yaml.

4. Run the server using cabal-install. The server will connect to the database in the cluster.

   cabal run back:exe:back

Images

Build Docker images, push to Docker Hub and pin image digests for app and test.

1. Create secrets.env file from secrets.example.env.

2. source environment files.

   1. Manually

      source secrets.env
      source .env

   2. Via direnv

      direnv allow

3. Push images to Docker Hub.

   nix run .#backPushToDockerHub
   nix run .#testPushToDockerHub

4. Write image digests.

   nix run .#writeDigests

Expose the server

1. The server IP address (let's call it <IP>) is localhost.

2. Get the port (let's call it <port>) of the server.

   • For the server in the cluster, see dev:back.service.nodePort in Pulumi.dev.yaml.
   • For the server run via cabal-install, see web.port in back.dev.yaml.

3. Expose the server to the world via localtunnel.

   lt -l <IP> -p <port> -s <subdomain>
   
   # or, in case of a cluster
   
   lt -l localhost -p 30003 -s breaking-news-fun

4. Copy the address (let's call it <address>) obtained from localtunnel.

Swagger

View the API, make requests to it.

1. Access the Swagger UI at <address>/api1/index.html.

Postman

View the API, make requests to it.

1. Import the API into Postman as a Postman Collection.

2. In that collection, set the variable baseUrl to <address>.

   1. Click on the collection -> Variables -> Current Value.
   2. Save.

3. Send a request to /api1/user/register.

   1. Set the values in the request Body.
   2. You'll get a refresh token (refreshToken) and an access token (accessToken).

4. Send a request to /api1/user/rotate-refresh-token.

   1. Copy refreshToken to Authorization -> BearerToken -> Token.
   2. You'll get a renewed pair of a refresh token and an access token.

5. Send the request again with the old refreshToken.

   1. You'll get a message that the session has a newer token.
   2. This is due to refresh token reuse detection (see Authorization).

6. Send a request to /api1/user/login.

   1. Copy the values from the request to /api1/user/register into Body.
   2. You'll get a renewed pair of a refresh token and an access token.

7. Send a request to api1/news/create.

   1. Copy the last accessToken to your Collection -> Variables -> {{bearerToken}}.
   2. Edit the value in the request Body.
   3. You'll get nothing in the response body.

8. Send a request to api1/news/get.

   1. Remove all query parameters.
   2. You'll get a list of news in the response body.

Database

1. Check the database.

   # connect to the database
   psql postgresql://admin:admin_password@localhost:30002/postgresdb
   
   # check records
   select * from users;

Monitoring

1. The cluster provides pod monitoring via loki-stack.

2. Monitoring is configured in Pulumi.dev.yaml at dev:monitoring.

3. Grafana should be available at http://localhost:30005.

   • Username: admin
   • Password: admin_password

4. In Explore -> Log browser, type {app="back-deployment"}

Pulumi

Manage resources in a running cluster via pulumi (see Cluster).

1. Go to pulumi.

   cd pulumi

2. Get urns.

   pulumi stack --show-urns

3. Try deleting a resource.

   pulumi state delete <urn>
   

4. If you delete a resource via kubectl, refresh pulumi. Usually, pulumi will notice the changes.

   # delete some pod
   kubectl delete po ...
   
   # after deleting resources, run
   pulumi refresh
   
   # update resources
   pulumi up

5. Destroy all resources.

   pulumi destroy

Configs

• package.yaml - used by hpack to generate a .cabal
• .markdownlint.jsonc - for markdownlint from the extension davidanson.vscode-markdownlint
• .ghcid - for ghcid
• .envrc - for direnv
• fourmolu.yaml - for fourmolu
• ci.yaml - a generated GitHub Actions workflow. See workflows. Generate a workflow via nix run .#writeWorkflows.
• hie.yaml - not present, but can be generated via implicit-hie (available on devshell) to verify the Haskell Language Server setup.

References

• charts/loki-stack
  • It's possible to overwrite chart values
    • Find grafana in requirements.yaml
    • Find its grafana chart values
    • Edit loki-stack values for grafana
• Grafana, Loki, Promtail - YT
  • similar
• DeriveAnyClass sometimes leads to infinite loops - see ToHttpApiData
• Running server in a VM
  • Host shouldn't be Windows
  • Forward port to internal server - src
  • How to Setup Port Forward with NAT Network in Virtualbox - YT
  • Set up prerouting on guest - SO
  • Disable firewall on guest - sudo ufw disable
• kustomization.yaml
• configmap
• web app
• Understanding Kubernetes Objects
• Debugging Templates (helm)
  • helm lint --strict
• toYaml correct indentation
• Persistent Volumes
  • PVC is bound to a PV at runtime automatically
  • There's no connection between PVCs and PVs in a manifest
• advanced helm templating
• Using production data in staging
• API versioning
  • via url part (easy to version the whole API)
• PostgreSQL Initialization scripts