Skip to content
This repository has been archived by the owner on Nov 25, 2022. It is now read-only.

Complement: Add Synapse Workers build step #123

Closed
wants to merge 3 commits into from
Closed

Complement: Add Synapse Workers build step #123

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2530a03
Complement: Add Synapse Workers build step
anoadragon453 Jan 21, 2021
51d459c
Move COMPLEMENT_BASE_IMAGE to environment:, fix dupl line
anoadragon453 Jan 25, 2021
d293d58
Remove another instance of this duplicate comment
anoadragon453 Jan 25, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions complement/pipeline.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,53 @@ steps:
- exit_status: 128
limit: 3

- command:
# Build the Synapse Workers docker image, which is located at:
# https://github.com/matrix-org/synapse/blob/master/docker/Docker-workers
# Note that this image is based on matrixdotorg/synapse:latest, which we're just
# pulling from docker hub
- wget https://github.com/matrix-org/synapse/archive/master.tar.gz
- tar -xzf master.tar.gz
Comment on lines +79 to +84
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we really need the whole of the synapse repo? If it's just the dockerfile you're after, just curl that?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately the dockerfile depends on additional files, named docker/conf/worker.yaml.j2 and docker/configure_workers_and_start.py.

It's probably easier to download the whole repo instead of the three files individually (which may become more files in the future).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fair enough.

Note that this incantation won't work until the dockerfile in question is present on synapse master.

Copy link
Member

@richvdh richvdh Jan 25, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On which note:

I have confirmed that this pipeline succeeds locally.

are you sure?

- docker build -t matrixdotorg/synapse:workers -f synapse-master/docker/Dockerfile-workers synapse-master/
# Build the accompanying Synapse workers Complement docker container. This specifically
# sets up a Synapse worker setup for Complement, and is located at:
# https://github.com/matrix-org/complement/blob/master/dockerfiles/SynapseWorkers.Dockerfile.
- docker build -t complement-synapse -f dockerfiles/SynapseWorkers.Dockerfile dockerfiles/
# Run the tests!
- COMPLEMENT_BASE_IMAGE=complement-synapse go test -v -tags synapse_blacklist ./tests
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this script could go in the complement repo, though it's not necessarily long enough to be worthwhile.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to keep it in here for now, so we have one place for the steps in the Complement pipeline and Synapse pipeline.

The wget and tar steps could probably be combined into one line as well. If it starts ballooning then moving it sounds more plausible.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to keep it in here for now, so we have one place for the steps in the Complement pipeline and Synapse pipeline.

well, they're in separate pipeline files, so I'm not sure that's much of a reason. Still, it's fine for now.

anoadragon453 marked this conversation as resolved.
Show resolved Hide resolved
label: "\U0001F9EA Complement / Synapse Workers / :go: 1.15"
agents:
# Running every worker takes a beefy system
queue: "xlarge"
plugins:
- docker#v3.7.0:
# The dockerfile for this image is at https://github.com/matrix-org/complement/blob/master/dockerfiles/ComplementCIBuildkite.Dockerfile.
image: "matrixdotorg/complement:latest"
mount-buildkite-agent: false
# Complement needs to know if it is running under CI
anoadragon453 marked this conversation as resolved.
Show resolved Hide resolved
environment:
# Complement needs to know if it is running under CI
- "CI=true"
# Enable debug logging
- "COMPLEMENT_DEBUG=1"
# Enabled Complement's certificate authority for authentication of
# federation requests
- "COMPLEMENT_CA=true"
# Starting up all the processes in this container can take a little while.
# The default is 200. Let's wait a bit longer to decrease the chance of a
# startup timeout
- "COMPLEMENT_VERSION_CHECK_ITERATIONS=500"
publish: [ "8448:8448" ]
# Complement uses Docker so pass through the docker socket. This means Complement shares
# the host's Docker.
volumes:
- "/var/run/docker.sock:/var/run/docker.sock"
retry:
automatic:
- exit_status: 128
limit: 3


- command:
# Build the Dendrite for Complement docker image, which is located at:
# https://github.com/matrix-org/complement/blob/master/dockerfiles/Dendrite.Dockerfile.
Expand Down