Skip to content

Commit a4aef8b

Browse files
donny-dontdonny-dont
committed
Add buildbot-worker documentation
Create documentation on how to connect a `buildbot-worker` container to WebKit infrastructure.
1 parent 9087bea commit a4aef8b

File tree

1 file changed

+142
-0
lines changed

1 file changed

+142
-0
lines changed

‎BUILDBOT.md‎

Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
# Connecting a WebKit Buildbot
2+
The `docker-webkit-dev` project contains Docker images for running Buildbot
3+
workers that connect to WebKit infrastructure. The Windows WebKit port builds
4+
happen within Docker containers running the `webkitdev/buildbot-worker` image.
5+
6+
> [!IMPORTANT]
7+
> To connect to WebKit infrastructure credentials are required.
8+
> Contact the infrastructure team on [Slack](https://webkit.slack.com) by
9+
> dropping a message in the `#dev` channel for assistance in connecting new
10+
> bots.
11+
12+
## Getting the image
13+
The `webkitdev/buildbot-worker` image is built and pushed to
14+
[Docker Hub](https://hub.docker.com/r/webkitdev/buildbot-worker). The following
15+
tags are supported. For the latest information on Windows container version
16+
compatibility see the
17+
[documentation](https://learn.microsoft.com/en-us/virtualization/windowscontainers/deploy-containers/version-compatibility)
18+
19+
| Tag Name | Automated | Description |
20+
|---|:---:|---|
21+
| 2022 | :white_check_mark: | A Windows 2022 server container |
22+
| windows-2022 | :x: | A Windows container, used for Layout Tests |
23+
| 1809 | :x: | A Windows 2019 server container |
24+
25+
Visit [Docker Hub](https://hub.docker.com/r/webkitdev/buildbot-worker/tags) to
26+
see when the last build happened for the image's tags.
27+
28+
## Configuring the Buildbot worker
29+
The `webkitdev/buildbot-worker` image uses environment variables for
30+
configuration. When invoking `docker run` environment variables are specified on
31+
the command line individually or through a file. See the
32+
[documentation](https://docs.docker.com/engine/reference/commandline/container_run/#env)
33+
for more details.
34+
35+
| Environment Variable | Required | Description |
36+
|---|:---:|---|
37+
| BUILD_HOST_NAME | :white_check_mark: | Host of the buildbot instance, either `build.webkit.org` or `ews-build.webkit.org` |
38+
| BUILD_HOST_PORT | :x: | Port of the buildbot instance, defaults to 17000 |
39+
| BUILD_WORKER_NAME | :white_check_mark: | Name for the worker, provided by infrastructure team |
40+
| BUILD_WORKER_PASSWORD | :white_check_mark: | Password for the worker, provided by infrastructure team |
41+
| BUILD_WORKER_KEEPALIVE | :x: | Time in seconds that messages should be sent by the worker to the server, defaults to 240 |
42+
| ADMIN_NAME | :x:| Contact name for the admin of the worker |
43+
| ADMIN_EMAIL | :x: | Contact e-mail address for the admin of the worker |
44+
| HOST_DESCRIPTION |:x: | A description of the host running the worker |
45+
| COMPILER | :x: | Needs to be set to `Clang` as the default `cl` is no longer able to build WebKit |
46+
47+
The preferred method of setup uses an environment file. An example file to fill
48+
out with all the options present looks like this. Save the text file locally
49+
with the name corresponding to worker name; for this example `build.env` is
50+
used.
51+
52+
```txt
53+
ADMIN_EMAIL=<email>
54+
ADMIN_NAME=<admin>
55+
BUILD_HOST_NAME=<host>
56+
BUILD_WORKER_NAME=<name>
57+
BUILD_WORKER_PASSWORD=<password>
58+
HOST_DESCRIPTION=<description>
59+
COMPILER=Clang
60+
```
61+
62+
## Running the Buildbot worker
63+
On a Windows Server 2022 machine run the following in a powershell session. On
64+
Windows Server 2022 the Docker container runs in process isolation mode which
65+
gives it access to all the resources of the host.
66+
67+
```powershell
68+
docker run --name build --env-file build.env --detach --restart always webkitdev/buildbot-worker:<tag>
69+
```
70+
71+
> [!IMPORTANT]
72+
> The worker will fail to start if not given enough resources to perform a
73+
> build. This limitation arises because, by default, the Windows container
74+
> allocates only two processors, which is inadequate for running a build. The
75+
> minimum requirements are 4 CPUs with 12GB of memory. However its best to give
76+
> as many CPUs as possible with each assigned at least 2GB of memory.
77+
78+
On a Windows 11 machine run the following in a powershell session. On Windows 11
79+
the Docker container runs in Hyper-V isolation mode so the amount of resources
80+
need to be specified with `cpu-count` and `memory`. Replace `X` with the number
81+
of processors to use and `Y` with the amount of memory in GBs to reserve for the
82+
container.
83+
84+
```powershell
85+
docker run --name build --env-file build.env --detach --restart always --cpu-count X --memory Yg webkitdev/buildbot-worker:<tag>
86+
```
87+
88+
To see if the container is running invoke the following in a powershell session.
89+
If everything is configured the output will look like this. Check the Buildbot
90+
instance and the worker should be present in its listing.
91+
92+
```powershell
93+
docker logs build
94+
95+
Buildbot information
96+
Name: <name>
97+
Admin: <admin> <email>
98+
Description: <description>
99+
Host system
100+
Processors: 1
101+
Logical processors: 16
102+
Total Physical Memory: 24.50gb
103+
\\60A8A235F96C\root\cimv2:Win32_LogicalDisk.DeviceID="C:"
104+
Disk information C:
105+
Total Disk Space: 126.87gb
106+
Available Disk Space: 126.74gb
107+
Initializing Visual Studio environment
108+
**********************************************************************
109+
** Visual Studio 2022 Developer Command Prompt v17.8.5
110+
** Copyright (c) 2022 Microsoft Corporation
111+
**********************************************************************
112+
[vcvarsall.bat] Environment initialized for: 'x64'
113+
Found compiler at C:\LLVM\bin\clang-cl.exe
114+
Initializing buildbot configuration
115+
Looking in C:\BW\Scripts for additional startup scripts
116+
0 scripts found
117+
Starting buildbot
118+
buildbot-worker start
119+
```
120+
121+
## Debugging
122+
To check on what is going on inside the container and diagnose an issue open a
123+
command prompt on the machine running the Docker container and enter the
124+
following command to get an interactive Powershell prompt within it.
125+
126+
```powershell
127+
docker exec -it build powershell
128+
```
129+
130+
An administrator Powershell instance is now running in the specified container,
131+
`build`, providing full access to it.
132+
133+
An example administrative task would be looking at the Buildbot logs within the
134+
container to diagnose a connection issue. To do that run the following command.
135+
136+
```powershell
137+
PS C:\BW> Get-Content .\twistd.log
138+
```
139+
140+
Whenever the task is over just use `exit` to terminate the session. As long as
141+
the container is running it can be re-entered using `docker exec`. Use it
142+
liberally to diagnose issues with the build.

0 commit comments

Comments
 (0)