Skip to content

Commit 1480a89

Browse files
donny-dontdonny-dont
authored
Add buildbot-worker documentation (#334)
Create documentation on how to connect a `buildbot-worker` container to WebKit infrastructure.
1 parent e5a355f commit 1480a89

File tree

1 file changed

+147
-0
lines changed

1 file changed

+147
-0
lines changed

‎BUILDBOT.md‎

Lines changed: 147 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,147 @@
1+
# Connecting a WebKit Buildbot
2+
The `docker-webkit-dev` project contains Docker images for running Buildbot
3+
workers that connect to WebKit infrastructure. The Windows WebKit port builds
4+
happen within Docker containers running the `webkitdev/buildbot-worker` image.
5+
6+
> [!IMPORTANT]
7+
> To connect to WebKit infrastructure credentials are required.
8+
> Contact the infrastructure team on [Slack](https://webkit.slack.com) by
9+
> dropping a message in the `#dev` channel for assistance in connecting new
10+
> bots.
11+
12+
## Getting the image
13+
The `webkitdev/buildbot-worker` image is built and pushed to
14+
[Docker Hub](https://hub.docker.com/r/webkitdev/buildbot-worker). The following
15+
tags are supported. For the latest information on Windows container version
16+
compatibility see the
17+
[documentation](https://learn.microsoft.com/en-us/virtualization/windowscontainers/deploy-containers/version-compatibility)
18+
19+
| Tag Name | Automated | Description |
20+
|---|:---:|---|
21+
| 2022 | :white_check_mark: | A Windows 2022 server container |
22+
| windows-2022 | :x: | A Windows container, used for Layout Tests |
23+
| 1809 | :x: | A Windows 2019 server container |
24+
25+
Visit [Docker Hub](https://hub.docker.com/r/webkitdev/buildbot-worker/tags) to
26+
see when the last build happened for the image's tags.
27+
28+
## Configuring the Buildbot worker
29+
The `webkitdev/buildbot-worker` image uses environment variables for
30+
configuration. When invoking `docker run` environment variables are specified on
31+
the command line individually or through a file. See the
32+
[documentation](https://docs.docker.com/engine/reference/commandline/container_run/#env)
33+
for more details.
34+
35+
| Environment Variable | Required | Description |
36+
|---|:---:|---|
37+
| BUILD_HOST_NAME | :white_check_mark: | Host of the buildbot instance, either `build.webkit.org`, for repository commits, or `ews-build.webkit.org` for pull requests |
38+
| BUILD_HOST_PORT | :x: | Port of the buildbot instance, defaults to 17000 |
39+
| BUILD_WORKER_NAME | :white_check_mark: | Name for the worker, provided by infrastructure team |
40+
| BUILD_WORKER_PASSWORD | :white_check_mark: | Password for the worker, provided by infrastructure team |
41+
| BUILD_WORKER_KEEPALIVE | :x: | Time in seconds that messages should be sent by the worker to the server, defaults to 240 |
42+
| ADMIN_NAME | :x:| Contact name for the admin of the worker |
43+
| ADMIN_EMAIL | :x: | Contact e-mail address for the admin of the worker |
44+
| HOST_DESCRIPTION |:x: | A description of the host running the worker |
45+
| COMPILER | :white_check_mark: | Needs to be set to `Clang` since the alternative, `cl`, is no longer able to build WebKit |
46+
47+
The preferred method of setup uses an environment file. An example file to fill
48+
out with all the options present looks like this. Save the text file locally
49+
with the name corresponding to worker name; for this example `build.env` is
50+
used.
51+
52+
```txt
53+
ADMIN_EMAIL=<email>
54+
ADMIN_NAME=<admin>
55+
BUILD_HOST_NAME=<host>
56+
BUILD_WORKER_NAME=<name>
57+
BUILD_WORKER_PASSWORD=<password>
58+
HOST_DESCRIPTION=<description>
59+
COMPILER=Clang
60+
```
61+
62+
## Running the Buildbot worker
63+
On a Windows Server 2022 machine run the following in a powershell session. On
64+
Windows Server 2022 the Docker container runs in process isolation mode which
65+
gives it access to all the resources of the host.
66+
67+
> [!NOTE]
68+
> Replace `<tag>` with the value from the above [table](#getting-the-image)
69+
> based on the type of worker being created.
70+
71+
```powershell
72+
docker run --name build --env-file build.env --detach --restart always --storage-opt size=80G webkitdev/buildbot-worker:<tag>
73+
```
74+
75+
> [!IMPORTANT]
76+
> The worker will fail to start if not given enough resources to perform a
77+
> build. This limitation arises because, by default, the Windows container
78+
> allocates only two processors, which is inadequate for running a build. The
79+
> minimum requirements are 4 CPUs with 12GB of memory and 30GB of storage.
80+
> However its best to give as many CPUs as possible with each assigned at least
81+
> 2GB of memory and 80GB of storage.
82+
83+
On a Windows 11 machine run the following in a powershell session. On Windows 11
84+
the Docker container runs in Hyper-V isolation mode so the amount of resources
85+
need to be specified with `cpu-count` and `memory`. Replace `X` with the number
86+
of processors to use and `Y` with the amount of memory in GBs to reserve for the
87+
container.
88+
89+
```powershell
90+
docker run --name build --env-file build.env --detach --restart always --storage-opt size=80G --cpu-count X --memory Yg webkitdev/buildbot-worker:<tag>
91+
```
92+
93+
To see if the container is running invoke the following in a powershell session.
94+
If everything is configured the output will look like this. Check the Buildbot
95+
instance and the worker should be present in its listing.
96+
97+
```powershell
98+
docker logs build
99+
100+
Buildbot information
101+
Name: <name>
102+
Admin: <admin> <email>
103+
Description: <description>
104+
Host system
105+
Processors: 1
106+
Logical processors: 16
107+
Total Physical Memory: 24.50gb
108+
\\60A8A235F96C\root\cimv2:Win32_LogicalDisk.DeviceID="C:"
109+
Disk information C:
110+
Total Disk Space: 126.87gb
111+
Available Disk Space: 126.74gb
112+
Initializing Visual Studio environment
113+
**********************************************************************
114+
** Visual Studio 2022 Developer Command Prompt v17.8.5
115+
** Copyright (c) 2022 Microsoft Corporation
116+
**********************************************************************
117+
[vcvarsall.bat] Environment initialized for: 'x64'
118+
Found compiler at C:\LLVM\bin\clang-cl.exe
119+
Initializing buildbot configuration
120+
Looking in C:\BW\Scripts for additional startup scripts
121+
0 scripts found
122+
Starting buildbot
123+
buildbot-worker start
124+
```
125+
126+
## Debugging
127+
To check on what is going on inside the container and diagnose an issue open a
128+
command prompt on the machine running the Docker container and enter the
129+
following command to get an interactive Powershell prompt within it.
130+
131+
```powershell
132+
docker exec -it build powershell
133+
```
134+
135+
An administrator Powershell instance is now running in the specified container,
136+
`build`, providing full access to it.
137+
138+
An example administrative task would be looking at the Buildbot logs within the
139+
container to diagnose a connection issue. To do that run the following command.
140+
141+
```powershell
142+
PS C:\BW> Get-Content .\twistd.log
143+
```
144+
145+
Whenever the task is over just use `exit` to terminate the session. As long as
146+
the container is running it can be re-entered using `docker exec`. Use it
147+
liberally to diagnose issues with the build.

0 commit comments

Comments
 (0)