Skip to content
Frank Rosner edited this page Nov 8, 2017 · 15 revisions

Instance Persistence

Broccoli is persisting the instances you create. You can select the desired storage back-end depending on your needs. Currently, there are two storage back-ends supported: fs and couchdb.

You can select which one you like by setting broccoli.instances.storage.type to either fs or couchdb. Please keep in mind that both might need additional configuration.

File System Storage fs

Description

If you want to try Broccoli, you can start with the file system based storage. It requires you to specify a directory where it can store instances as JSON files through broccoli.instances.storage.fs.url.

On startup, Broccoli will lock this folder (by creating a .lock file inside) because the file system back-end does not support concurrent access on these files. In case your Broccoli terminated irregularly you might need to check that you have no other Broccoli accessing this folder and delete the .lock file manually before restarting. Watch the logs if any data got corrupted or all instances could be loaded correctly.

Configuration

broccoli.instances.storage.type=fs
broccoli.instances.storage.fs.url=instances

CouchDB Storage couchdb

Description

For a production setup you can use the CouchDB back-end. Also it is guaranteed that you don't end up with corrupted data if Broccoli dies in the middle of a transaction. Note that we only support CouchDB 1.7 at the moment.

On startup, Broccoli will check whether the specified database is reachable. If it does not exist it will attempt to create it.

Although it would technically be possible to point two Broccoli instances to the same CouchDB database, it is not recommended because one instance will not see the changes that another instance is making. This is why Broccoli is also locking the CouchDB database and refuses to start if another Broccoli is using it already. Locking is done by creating a config entry at /_config/broccoli/<db_name>.

Configuration

broccoli.instances.storage.type=couchdb
broccoli.instances.storage.couchdb.url=http://localhost:5984
broccoli.instances.storage.couchdb.dbName=broccoli_instances

Job and Service Status

Job and service status are obtained periodically from Nomad and Consul. They are cached in-memory inbetween updates and are not persisted. If Nomad or Consul are not reachable, the state will also not be available on Broccoli as soon as the cache gets invalidated.

You can control the time between two requests for job and service status by setting broccoli.polling.frequency (in seconds).

Migrating Instances

When you want to migrate instances from one Broccoli to another you have two options.

  1. You migrate the storage (e.g. duplicate the CouchDB database or copy the folder of the file system storage).
  2. You start both Broccoli instances and use the broccoli2broccoli.js script. Note that this will not preserve instances referencing outdated templates but will attempt to create them with the current version of the templates.