Skip to content

Commit

Permalink
Simplify and refactor docker-compose.yml and remove install scripts.
Browse files Browse the repository at this point in the history 
- Remove "demo", "production" containers and have just one.
- Remove dependency on config.toml and inline config to env vars.
- Have a single idempotent instal+upgrade+run command that eliminates
  the need for manual upgrade steps.
- Remove `.sh` install scripts.
- Simplify and clean up install and update docs.
  • Loading branch information
@knadh
knadh committed Oct 27, 2024
1 parent 24bab75 commit 7fcc6f7
Show file tree
Hide file tree
Showing 7 changed files with 121 additions and 376 deletions.
32 changes: 12 additions & 20 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,38 +12,30 @@ Visit [listmonk.app](https://listmonk.app) for more info. Check out the [**live

### Docker

The latest image is available on DockerHub at [`listmonk/listmonk:latest`](https://hub.docker.com/r/listmonk/listmonk/tags?page=1&ordering=last_updated&name=latest). Use the sample [docker-compose.yml](https://github.com/knadh/listmonk/blob/master/docker-compose.yml) to run manually or use the helper script.
The latest image is available on DockerHub at [`listmonk/listmonk:latest`](https://hub.docker.com/r/listmonk/listmonk/tags?page=1&ordering=last_updated&name=latest).
Download and use the sample [docker-compose.yml](https://github.com/knadh/listmonk/blob/master/docker-compose.yml).

#### Demo

```bash
mkdir listmonk-demo && cd listmonk-demo
bash -c "$(curl -fsSL https://raw.githubusercontent.com/knadh/listmonk/master/install-demo.sh)"
```

DO NOT use this demo setup in production.
```shell
# Download the compose file to the current directory.
curl -O https://github.com/knadh/listmonk/blob/master/docker-compose.yml

#### Production

```bash
mkdir listmonk && cd listmonk
bash -c "$(curl -fsSL https://raw.githubusercontent.com/knadh/listmonk/master/install-prod.sh)"
# Run the services in the background.
docker compose up -d
```
Visit `http://localhost:9000`.

**NOTE**: Always examine the contents of shell scripts before executing them.
Visit `http://localhost:9000`

See [installation docs](https://listmonk.app/docs/installation).
See [installation docs](https://listmonk.app/docs/installation)

__________________

### Binary
- Download the [latest release](https://github.com/knadh/listmonk/releases) and extract the listmonk binary.
- `./listmonk --new-config` to generate config.toml. Then, edit the file.
- `./listmonk --new-config` to generate config.toml. Edit it.
- `./listmonk --install` to setup the Postgres DB (or `--upgrade` to upgrade an existing DB. Upgrades are idempotent and running them multiple times have no side effects).
- Run `./listmonk` and visit `http://localhost:9000`.
- Run `./listmonk` and visit `http://localhost:9000`

See [installation docs](https://listmonk.app/docs/installation).
See [installation docs](https://listmonk.app/docs/installation)
__________________


Expand Down
103 changes: 52 additions & 51 deletions docker-compose.yml
View file
Original file line number Diff line number Diff line change
@@ -1,64 +1,65 @@
# NOTE: This docker-compose.yml is an example setup.
# It is not intented to run out of the box
# and you must edit the below configurations to suit your needs.
# See https://listmonk.app/docs/installation/#manual-docker-install_1
x-db-credentials: &db-credentials # Use the default POSTGRES_ credentials if they're available or simply default to "listmonk"
POSTGRES_USER: &db-user listmonk # for database user, password, and database name
POSTGRES_PASSWORD: &db-password listmonk
POSTGRES_DB: &db-name listmonk

x-app-defaults: &app-defaults
restart: unless-stopped
image: listmonk/listmonk:latest
ports:
- "9000:9000"
networks:
- listmonk
environment:
- TZ=Etc/UTC
services:
# listmonk app
app:
image: listmonk/listmonk:test
container_name: listmonk_app
restart: unless-stopped
ports:
- "9000:9000" # To change the externally exposed port, change to: $custom_port:9000
networks:
- listmonk
hostname: listmonk.example.com # Recommend using FQDN for hostname
depends_on:
- db
command: [sh, -c, "./listmonk --install --idempotent --yes --config '' && ./listmonk --upgrade --yes --config '' && ./listmonk --config ''"]
# --config (file) param is set to empty so that listmonk only uses the env vars (below) for config.
# --install --idempotent ensures that DB installation happens only once on an empty DB, on the first ever start.
# --upgrade automatically runs any DB migrations when a new image is pulled.

x-db-defaults: &db-defaults
image: postgres:14-alpine
ports:
- "9432:5432"
networks:
- listmonk
environment:
- POSTGRES_PASSWORD=listmonk
- POSTGRES_USER=listmonk
- POSTGRES_DB=listmonk
restart: unless-stopped
healthcheck:
test: ["CMD-SHELL", "pg_isready -U listmonk"]
interval: 10s
timeout: 5s
retries: 6
environment: # The same params as in config.toml are passed as env vars here.
LISTMONK_app__address: 0.0.0.0:9000
LISTMONK_db__user: *db-user
LISTMONK_db__password: *db-password
LISTMONK_db__database: *db-name
LISTMONK_db__host: listmonk_db
LISTMONK_db__port: 5432
LISTMONK_db__ssl_mode: disable
LISTMONK_db__max_open: 25
LISTMONK_db__max_idle: 25
LISTMONK_db__max_lifetime: 300s
TZ: Etc/UTC
LISTMONK_ADMIN_USER: ${LISTMONK_ADMIN_USER:-} # If these (optional) are set during the first `docker compose up`, then the Super Admin user is automatically created.
LISTMONK_ADMIN_PASSWORD: ${LISTMONK_ADMIN_PASSWORD:-} # Otherwise, the user can be setup on the web app after the first visit to http://localhost:9000
volumes:
- ./uploads:/listmonk/uploads:rw # Mount an uploads directory on the host to /listmonk/uploads inside the container.
# To use this, change directory path in Admin -> Settings -> Media to /listmonk/uploads

services:
# Postgres database
db:
<<: *db-defaults
image: postgres:17-alpine
container_name: listmonk_db
restart: unless-stopped
ports:
- "5432:5432"
networks:
- listmonk
environment:
<<: *db-credentials
healthcheck:
test: ["CMD-SHELL", "pg_isready -U listmonk"]
interval: 10s
timeout: 5s
retries: 6
volumes:
- type: volume
source: listmonk-data
target: /var/lib/postgresql/data

app:
<<: *app-defaults
container_name: listmonk_app
hostname: listmonk.example.com # Recommend using FQDN for hostname
depends_on:
- db
volumes:
- ./config.toml:/listmonk/config.toml

demo-db:
container_name: listmonk_demo_db
<<: *db-defaults

demo-app:
<<: *app-defaults
container_name: listmonk_demo_app
command: [sh, -c, "yes | ./listmonk --install --config config-demo.toml && ./listmonk --config config-demo.toml"]
depends_on:
- demo-db

networks:
listmonk:

Expand Down
128 changes: 36 additions & 92 deletions docs/docs/content/installation.md
View file
Original file line number Diff line number Diff line change
@@ -1,105 +1,54 @@
# Installation

listmonk requires Postgres ⩾ 12

!!! Admin
listmonk generates and prints admin credentials to the terminal during installation. This can be copied to login to the admin dashboard and later changed. To choose a custom username and password during installation,
set the environment variables `LISTMONK_ADMIN_USER` and `LISTMONK_ADMIN_PASSWORD` during installation.
listmonk is a simple binary application that requires a Postgres database instance to run. The binary can be downloaded and run manually, or it can be run as a container with Docker compose.

## Binary
1. Download the [latest release](https://github.com/knadh/listmonk/releases) and extract the listmonk binary. `amd64` is the main one. It works for Intel and x86 CPUs.
1. `./listmonk --new-config` to generate config.toml. Edit the file.
1. `./listmonk --install` to install the tables in the Postgres DB. Copy the admin username and password from the terminal output (these can be changed from the admin UI later). To choose a custom username and password during installation, run: `LISTMONK_ADMIN_USER=myuser LISTMONK_ADMIN_PASSWORD=xxxxx ./listmonk --install`
1. Run `./listmonk` and visit `http://localhost:9000`.
1. `./listmonk --install` to install the tables in the Postgres DB (⩾ 12).
1. Run `./listmonk` and visit `http://localhost:9000` to create the Super Admin user and login.

!!! Tip
To set the Super Admin username and password during installation, set the environment variables:
`LISTMONK_ADMIN_USER=myuser LISTMONK_ADMIN_PASSWORD=xxxxx ./listmonk --install`


## Docker

The latest image is available on DockerHub at `listmonk/listmonk:latest`

!!! note
listmonk's docs and scripts use `docker compose`, which is compatible with the latest version of docker. If you installed docker and docker-compose from your Linux distribution, you probably have an older version and will need to use the `docker-compose` command instead, or you'll need to update docker manually. [More info](https://gist.github.com/MaximilianKohler/e5158fcfe6de80a9069926a67afcae11#docker-update).

Use the sample [docker-compose.yml](https://github.com/knadh/listmonk/blob/master/docker-compose.yml) to run listmonk and Postgres DB with `docker compose` as follows:

### Demo

#### Easy Docker install

```bash
mkdir listmonk-demo && cd listmonk-demo
bash -c "$(curl -fsSL https://raw.githubusercontent.com/knadh/listmonk/master/install-demo.sh)"
```

#### Manual Docker install
The recommended method is to download the [docker-compose.yml](https://github.com/knadh/listmonk/blob/master/docker-compose.yml) file, customize it for your environment and then to simply run `docker compose up -d`.

```bash
wget -O docker-compose.yml https://raw.githubusercontent.com/knadh/listmonk/master/docker-compose.yml
docker compose up -d demo-db demo-app
```

!!! warning
The demo does not persist Postgres after the containers are removed. **DO NOT** use this demo setup in production.

### Production

#### Easy Docker install

This setup is recommended if you want to _quickly_ setup `listmonk` in production.
```shell
# Download the compose file to the current directory.
curl -O https://github.com/knadh/listmonk/blob/master/docker-compose.yml

```bash
mkdir listmonk && cd listmonk
bash -c "$(curl -fsSL https://raw.githubusercontent.com/knadh/listmonk/master/install-prod.sh)"
# Run the services in the background.
docker compose up -d
```

The above shell script performs the following actions:

- Downloads `docker-compose.yml` and generates a `config.toml`.
- Runs a Postgres container and installs the database schema.
- Runs the `listmonk` container.
Then, visit `http://localhost:9000` to create the Super Admin user and login.

!!! note
It's recommended to examine the contents of the shell script, before running in your environment.
!!! Tip
To set the Super Admin username and password during setup, set the environment variables (only the first time):
`LISTMONK_ADMIN_USER=myuser LISTMONK_ADMIN_PASSWORD=xxxxx docker compose up -d`

#### Manual Docker install

The following workflow is recommended to setup `listmonk` manually using `docker compose`. You are encouraged to customise the contents of [`docker-compose.yml`](https://github.com/knadh/listmonk/blob/master/docker-compose.yml) to your needs. The overall setup looks like:
------------

- `docker compose up db` to run the Postgres DB.
- `docker compose run --rm app ./listmonk --install` to setup the DB (or `--upgrade` to upgrade an existing DB).
- Copy `config.toml.sample` to your directory and make the following changes:
- `app.address` => `0.0.0.0:9000` (Port forwarding on Docker will work only if the app is advertising on all interfaces.)
- `db.host` => `listmonk_db` (Container Name of the DB container)
- Run `docker compose up app` and visit `http://localhost:9000`.
### Mounting a custom config.toml
The docker-compose file includes all necessary listmonk configuration as environment variables, `LISTMONK_*`.
If you would like to remove those and mount a config.toml instead:

##### Mounting a custom config.toml

To mount a local `config.toml` file, add the following section to `docker-compose.yml`:

```yml
app:
<<: *app-defaults
depends_on:
- db
volumes:
- ./path/on/your/host/config.toml:/listmonk/config.toml
```
!!! note
Some common changes done inside `config.toml` for Docker based setups:

- Change `app.address` to `0.0.0.0:9000`.
- Change `db.host` to `listmonk_db`.

Here's a sample `config.toml` you can use:
#### 1. Save the config.toml file on the host

```toml
[app]
address = "0.0.0.0:9000"

# Database.
[db]
host = "listmonk_db"
host = "listmonk_db" # Postgres container name in the compose file.
port = 5432
user = "listmonk"
password = "listmonk"
Expand All @@ -110,25 +59,20 @@ max_idle = 25
max_lifetime = "300s"
```

Mount the local `config.toml` inside the container at `listmonk/config.toml`.
#### 2. Mount the config file in docker-compose.yml

!!! tip
- See [configuring with environment variables](configuration.md) for variables like `app.admin_password` and `db.password`
- Ensure that both `app` and `db` containers are in running. If the containers are not running, restart them `docker compose restart app db`.
- Refer to [this tutorial](https://yasoob.me/posts/setting-up-listmonk-opensource-newsletter-mailing/) for setting up a production instance with Docker + Nginx + LetsEncrypt SSL.

!!! info
The example `docker-compose.yml` file works with Docker Engine 24.0.5+ and Docker Compose version v2.20.2+.

##### Changing the port

To change the port for listmonk:
```yaml
app:
...
volumes:
- /path/on/your/host/config.toml:/listmonk/config.toml
```
- Ensure no other container of listmonk app is running. You can check with `docker ps | grep listmonk`.
- Change [L11](https://github.com/knadh/listmonk/blob/master/docker-compose.yml#L11) to `custom-port:9000` Eg: `3876:9000`. This will expose the port 3876 on your local network to the container's network interface on port 9000.
- For NGINX setup, if you're running NGINX on your local machine, you can proxy_pass to the `<MACHINE_IP>:3876`. You can also run NGINX as a docker container within the listmonk's container (for that you need to add a service `nginx` in the docker-compose.yml). If you do that, then proxy_pass will be set to `http://app:9000`. Docker's network will resolve the DNS for `app` and directly speak to port 9000 (which the app is exposing within its own network).

#### 3. Change the `--config ''` flags in the `command:` section to point to the path

```yaml
command: [sh, -c, "./listmonk --install --idempotent --yes --config /listmonk/config.toml && ./listmonk --upgrade --yes --config /listmonk/config.toml && ./listmonk --config /listmonk/config.toml"]
```


## Compiling from source
Expand All @@ -137,13 +81,13 @@ To compile the latest unreleased version (`master` branch):

1. Make sure `go`, `nodejs`, and `yarn` are installed on your system.
2. `git clone git@github.com:knadh/listmonk.git`
3. `cd listmonk && make dist`. This will generate the `listmonk binary`.
3. `cd listmonk && make dist`. This will generate the `listmonk` binary.

## Release candidate (RC)

The `master` branch with bleeding edge changes is periodically built and published as `listmonk/listmonk:rc` on DockerHub. To run the latest pre-release version, replace all instances of `listmonk/listmonk:latest` with `listmonk/listmonk:rc` in the docker-compose.yml file and follow the Docker installation steps above. While it is generally safe to run release candidate versions, they may have issues that only get resolved in a general release.

## Helm chart for kubernetes
## Helm chart for Kubernetes

![Version: 0.1.0](https://img.shields.io/badge/Version-0.1.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 3.0.0](https://img.shields.io/badge/AppVersion-3.0.0-informational?style=flat-square)

Expand Down
15 changes: 9 additions & 6 deletions docs/docs/content/upgrade.md
View file
Original file line number Diff line number Diff line change
@@ -1,21 +1,24 @@
# Upgrade

Some versions may require changes to the database. These changes or database "migrations" are applied automatically and safely, but, it is recommended to take a backup of the Postgres database before running the `--upgrade` option, especially if you have made customizations to the database tables.
!!! Warning
Always take a backup of the Postgres database before upgrading listmonk

## Binary
- Download the [latest release](https://github.com/knadh/listmonk/releases) and extract the listmonk binary.
- `./listmonk --upgrade` to upgrade an existing DB. Upgrades are idempotent and running them multiple times have no side effects.
- Run `./listmonk` and visit `http://localhost:9000`.
- Stop the running instance of listmonk.
- Download the [latest release](https://github.com/knadh/listmonk/releases) and extract the listmonk binary and overwrite the previous version.
- `./listmonk --upgrade` to upgrade an existing database schema. Upgrades are idempotent and running them multiple times have no side effects.
- Run `./listmonk` again.

If you installed listmonk as a service, you will need to stop it before overwriting the binary. Something like `sudo systemctl stop listmonk` or `sudo service listmonk stop` should work. Then overwrite the binary with the new version, then run `./listmonk --upgrade, and `start` it back with the same commands.

If it's not running as a service, `pkill -9 listmonk` will stop the listmonk process.

## Docker
> Instructions for versions above v4.x.x using the latest [docker-compose.yml](https://github.com/knadh/listmonk/blob/master/docker-compose.yml) file.
- `docker compose down app` and stop the app container.
- `docker compose pull` to pull the latest version from DockerHub.
- `docker compose run --rm app ./listmonk --upgrade` to upgrade an existing DB.
- Run `docker compose up app db` and visit `http://localhost:9000`.
- `docker compose up app -d` to automatically run the upgrade and start listmonk.

## Railway
- Head to your dashboard, and select your Listmonk project.
Expand Down
Loading

0 comments on commit 7fcc6f7

Please sign in to comment.