Scaling compute capacity?

[prologic]

Does OpenRun have or will have the capability to scale with additional machines / compute?

[akclace]

There are three planned deployment models for OpenRun
Option 1. Single node, using SQLite for metadata, no horizontal scaling, only vertical scaling
Option 2. Multiple nodes, behind a load balancer, user brings their postgres database for metadata
Option 3. Kubernetes deployment

Currently, the first one (sqlite based) is implemented. Postgres support has been implemented, still need to add some more tests and docs for the second option. Kubernetes based deployment has not been worked on yet, that would use a postgres db and an external container registry.

[prologic]

Have you considers a clustered architecture using Raft?

[akclace]

Have you considers a clustered architecture using Raft?

The multi node support uses postgres, since a DB is needed for metadata (apps, versions, audit logs etc). Things like leader election can be done in postgres, there is no plan to add Raft

[akclace]

Had a DM discussion with James. He wanted to avoid the Postgres dependency to reduce operation complexity, instead using Raft for leader election and something like Consul for metadata.

OpenRun uses a relational database (SQLite by default for single node, Postgres for multi node) for the following reasons/usecases:

1. Atomic updates - the transaction is rolled back if anything fails, allowing transactional updates across any number of apps
2. Dry-run support - all operations can be run in dry-run mode, allowing verification of what is going to happen without actually making any changes
3. Atomic updates for app file contents - see https://openrun.dev/blog/sqlite/
4. Audit operations use the SQL DB for high volume updates

Replacing the database with Consul/etcd would be non-trivial. For larger clusters, having a Postgres instance would not be a challenge, somethings like RDS would work fine. For smaller installations, managing a Postgres instance is an additional overhead.

For a single node installation using SQLite, a build node which registers with the api worker which takes care of container builds while not requiring a postgres database would be a partial solution. Multiple api workers would still require a Postgres database.