Skip to content

Commit

Permalink
Update README on how to install a new worker
Browse files Browse the repository at this point in the history
  • Loading branch information
ottok committed Sep 3, 2018
1 parent 7df6cce commit 7bc8469
Showing 1 changed file with 19 additions and 6 deletions.
25 changes: 19 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,15 +54,15 @@ buildbot.mariadb.org/

We are using Docker for targeting Linux builds. This makes it easy (and cheap) to add new distribution flavored builders with just a Dockerfile, while also facilitating environment replication locally, without much effort, making it easy to inspect, reproduce the build environment and debug failures.

Using Buildbot's DockerLatentWorker to connect to docker instances, the worker runs inside the Docker image, and on a build trigger, master connects to Docker first via docker-py , instantiates the image and then connects using Buildbot usual worker API to do the heavy lifting. Upon build completion, the image is stopped and no left-overs are left behind.
Using Buildbot's DockerLatentWorker to connect to docker instances, the worker runs inside the Docker image, and on a build trigger, master connects to Docker first via docker-py , instantiates the image and then connects using Buildbot usual Worker API to do the heavy lifting. Upon build completion, the image is stopped and no left-overs are left behind.


## Navigating the new interface
===============================

Starting with version 1.0, Buildbot interface changed in a significant and positive manner. The new UI is a fresh rewrite with responsive, lazy loading and modern web conveniences that makes it easy to follow running builds as well as convenient to browse build history.

Grid View and MTRLogObserver plugins are usable out of the box and behave in a similar fashion as in the current Buildbot instance.
Grid View and MTRLogObserver plugin are usable out of the box and behave in a similar fashion as in the current Buildbot instance.

One notable change is that, by default, new Buildbot does not load massive build/revision/branch/change historical data unless you increase the default numbers in _Settings_ -> _Grid related settings_ in your current browser session. 40 revisions, 1000 for changes & builds are good for a start. Play around with the values you see in Settings pane, they enable some of the behavior from current Buildbot, that you might be used to.

Expand All @@ -71,7 +71,7 @@ One notable change is that, by default, new Buildbot does not load massive build

Debugging build failures with Docker is facilitated by the ease of replicating the actual build environment where a particular builder failed. First step is to identify on which platform the failure occurred and what's the associated Dockerfile. If you're not familiar with Docker, start by reading the [Docker overview](https://docs.docker.com/engine/docker-overview/) and [Docker getting started documentation](https://docs.docker.com/get-started/).

To reproduce a build, first identify the platform and look for the associated dockerfile in the dockerfiles/ directory. To create a container for Ubuntu 18.04 for e.g. you would use the *docker build* command like this:
To repeat a build on buildbot, first identify the platform and look for the associated dockerfile in the dockerfiles/ directory. To create a container for Ubuntu 18.04 for e.g. you would use the *docker build* command like this:
`docker build -t ubuntu-1804 --file ubuntu1804.dockerfile .`

After a successful image build, you can run a container and execute bash using that image with *docker run*:
Expand Down Expand Up @@ -116,13 +116,26 @@ Also, master.cfg sources a private config file that is not included in this repo

### Reloading the master

For changes to the master.cfg to take effect, the server needs to be reloaded. While doing this, please keep an eye on the logs with `tail -f /srv/buildbot/master/twistd.log`. The Buildbot master is managed by our own rolled systemd file. To invoke a reload run `sudo systemctl reload buildbot-master`. The other usual systemctl are also available (`status`, `restart`). It is also considered good practice to check the changes you make for syntax or other problems with the `buildbot checkconfig master.cfg` command before restarting or reconfiguring the server.
For changes to the master.cfg to take effect, the server needs to be reloaded. While doing this, please keep an eye on the logs with `tail -f /srv/buildbot/master/twistd.log`. The Buildbot master is managed by our own rolled systemd file. To invoke a reload run `sudo systemctl reload buildbot-master`. The other usual systemctl are also available (`status`, `restart`). Before the reload, consider testing the config with `buildbot checkconfig`.

## Adding workers

There's currently two types of workers that we use. Normal Buildbot workers, running bare-bones builds and can be added using `c['workers'].append(mkWorker("bm-bbw1-ubuntu1804"))`.
There's currently two types of workers that we use. Normal Buildbot workers, running bare-bones builds and can be added using `c['workers'].append(mkWorker("bm-bbw1-ubuntu1804"))`. The password entry needs to be defined in `master-private.cfg`. Note that the worker name cannot be a hostname with dots. Only alphanumeric and dashes are allowed in the worker name.

For instructions on setting up a Buildbot worker see http://docs.buildbot.net/latest/manual/installation/worker.html.
For instructions on setting up a Buildbot Worker see http://docs.buildbot.net/latest/manual/installation/worker.html.

For example, on a Debian Stretch the following commands would be needed to set it up as a worker for our Buildbot:
```
adduser --gecos "Buildbot worker user" --disabled-password --disabled-login buildbot
apt install --yes python3-pip
pip3 install buildbot-worker
curl -IL buildbot.mariadb.org:9989 # Should respond with 'Empty reply from server' and not timeout
su - buildbot
buildbot-worker create-worker ~/worker buildbot.mariadb.org:9989 <worker name> <worker password>
nano ~/worker/info/* # Set admin email and worker description
buildbot-worker start --verbose ~/worker
exit
```

The other type of worker we use is [DockerLatentWorker](http://docs.buildbot.net/latest/manual/cfg-workers-docker.html). It is useful for running tests inside single use, disposable container Linux instances. We are currently aiming to use it for most of our Linux builds.

Expand Down

0 comments on commit 7bc8469

Please sign in to comment.