Skip to content
Matt Simerson edited this page May 15, 2024 · 26 revisions

Table of Contents

  1. How much RAM is needed?
  2. What has changed
  3. What features are planned?
  4. Where is my old jail?
  5. Why microservices?
  6. Why Jails?
  7. How to enable qmail-extensions?
  8. How can users change their password
  9. How can I enable lastauth updating

How much RAM is needed?

  • A lightly used server: 2GB. Use only one virus and spam scanner, reduce zfs.arc_max, set node to aggressively garbage collect.
  • A small server with a few hundred users: 4GB would be just right.
  • Everyone else: 8GB min, 16GB is great, more than 32 is future-proofing.

Large RAM consumers:

  • ZFS: 1+ GB (see vfs.zfs.arc_max)
  • haraka (node.js): 1GB (see --max_old_space_size)
  • clamd: 712MB
  • mysql: 600MB
  • AVG: 300MB
  • rspamd: 263MB
  • rspamd + phishing: 600MB
  • spamassassin: 200MB

A good cloud based server (VPS/VM/jail) that satisfies these requirements should run about $40/mo. A server which greatly exceeds these requirements (48GB RAM, 8 Xeon cores, 3-4 disks) runs about $400 on eBay.


What has changed

since the tnpi wiki instructions?

  • GitHub! (issue tracking & PRs)
  • jail lifecycle has shortened
    • jails are now ephemeral
    • instead of build-and-[upgrade/maintain]-each-jail, now it's build-use-rebuild
  • data filesystems
    • data that persists beyond the lifecycle of a jail lives in a data filesystem
    • examples: user mailboxes (/data/vpopmail), webmail prefs (/data/webmail).
  • build improvements
    • much less copy/paste!
    • more automation (plenty of "do ..." tasks scripted away)
    • fewer file downloads (replaced by here docs and build-as-we-go provisioning)
    • pkg caching (avoids redundant pkg downloads when provisioning)
  • dns
    • create an unbound.conf.local file in the provisioning directory for local customizations
  • networking
    • default jail network now 172.16.15 (was 127.0.0). (Configurable, see JAIL_NET_PREFIX)
    • default jail NIC now lo1 (was lo0)
  • dovecot
    • moved into its own jail (possible thanks to shared data filesystem)

What features are planned?

See the Roadmap


Where is my old jail

When you provision a new jail the old filesystem gets renamed to $name.last. It remains until you provision that jail again (or manually destroy it). If you provision a new jail and it doesn't work properly, you can always restore the last working version.


Why microservices

  • Each service (aka. deployment unit) is now tiny and easy to deploy.
  • No more upgrades! Deployments are fast and easy, so deploy newer jails and tear down old ones.
    • data persists on a dedicated data volume
    • just in case, the most recent service file system persists as $service.last (ex: dns.last, redis.last)
  • Easier administration:
    • Each VM has only the software it needs, reducing dependency chains.
    • the webmail apps and PHP are deployed in their own jail.
    • SpamAssassin and its many perl modules are in their own.
  • Consistent environments: fresh deployments (vs upgrades) keeps environments consistent and up-to-date. See Snowflake Server.
  • Flexibility:
    • skip / ignore services you don't want/need.
    • use nginx for PHP web services, lighttpd for CGI, and Apache for ... uh...legacy stuff.
  • Scalability: scale just the components that need it.
    • need greater SpamAssassin throughput? Tune it! Need more? Spin up another instance and load balance the requests.
    • move the mail store onto your SAN
    • put HAproxy on a pair of load balanced servers
  • Security: services are isolated into VMs and are now network services. Service isolation helps limit the ability of attackers to escalate vulnerabilities.

Why Jails

  • ZFS filesystems
  • jails are extremely cheap.
    • jails are far less resource intensive than a hypervisor (Xen, VMware, HyperV, Virtual Box)
    • thousands of jails can be added to a single physical machine.
    • jails are only a tiny bit more expensive than dockers
  • jails are more secure
    • jails provide sufficient isolation / multi-tenancy support
    • but not as much as a hypervisor
  • jails are simpler and faster to administer than full VMs
  • composability: jails can be created, destroyed and re-created in seconds, lending themselves to light weight thinly provisioned services
  • a single ports tree can be updated once (daily) and shared by all jails

Jails (and Solaris Containers) are between the high expense but excellent isolation of a hypervisor and the low expense but poor isolation of a docker / linux container. All great tools, with each serving their niches very, very well.


How to enable qmail-extensions

  • in the vpopmail jail, add $Qmail::Deliverable::VPOPMAIL_EXT = 1; to /usr/local/bin/qmail-deliverabled. Then kill the qmail-deliverabled daemon. It will be restarted automatically.

How To Update Password

  1. via qmailadmin
  2. via qmailadmin and the squirrelmail qmailadmin plugin
  3. roundcube webmail, via the password plugin. Configure for vpopmaild. Bonus: has password strength requirements.

How To Update Lastauth

See Dovecot-SQL-updating-mysql-lastauth

Clone this wiki locally