Skip to content

Conversation

@alco
Copy link
Member

@alco alco commented Dec 16, 2025

This reproduces our Buildkite pipeline on GitHub Actions and allows us to retire Buildkite.

The most painful problem that this solves are the missing webhooks that result in missed release images on Docker Hub. In a few words, when we push a release commit to main that touches 4 or more components, and so the commit itself gets 4+ tags, GitHub does not send a webhook to Buildkite because of its limit on max tags. In such cases, we always had to manually force-push only the @core/sync-service:... tag to trigger a Buildkite pipeline.

The GitHub Actions workflow implemented here is verbose because it has to orchestrate a single shared step, followed by a matrix that uses a separate runner for building different platforms, followed by another shared step that collects the digests from the matrix and combines them into a single multi-arch tagged image on Docker Hub.

It may be possible to simplify it later on, since our next target is migrating some or all of our runners to Blacksmith, and Blacksmith promises to implement implicit forking of Docker builder into different runners based on the passed platforms, kinda like Buildkite has been doing.

Closes #3607.

@codecov
Copy link

codecov bot commented Dec 16, 2025

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
1920 1 1919 0
View the top 2 failed test(s) by shortest run time
Elixir.Electric.Plug.RouterTest::test /v1/shapes - subqueries move-in into move-out into move-in of the same parent results in a 
Stack Traces | 1.32s run time
71) test /v1/shapes - subqueries move-in into move-out into move-in of the same parent results in a  (Electric.Plug.RouterTest)
     .../electric/plug/router_test.exs:2749
     match (=) failed
     code:  assert {_req, 200,
             [
               %{"headers" => %{"event" => "move-out", "patterns" => p1}},
               %{"headers" => %{"event" => "move-out", "patterns" => p1}},
               %{"headers" => %{"control" => "snapshot-end"}},
               %{
                 "headers" => %{"operation" => "insert", "is_move_in" => true},
                 "value" => %{"id" => "3", "parent_id" => "3", "value" => "30"}
               },
               %{"headers" => %{"control" => "snapshot-end"}},
               up_to_date_ctl()
             ]} = shape_req(req, ctx.opts)
     left:  {_req, 200,
             [
               %{"headers" => %{"event" => "move-out", "patterns" => p1}},
               %{"headers" => %{"event" => "move-out", "patterns" => p1}},
               %{"headers" => %{"control" => "snapshot-end"}},
               %{
                 "headers" => %{"operation" => "insert", "is_move_in" => true},
                 "value" => %{"id" => "3", "parent_id" => "3", "value" => "30"}
               },
               %{"headers" => %{"control" => "snapshot-end"}},
               up_to_date_ctl()
             ]}
     right: {%{handle: "89020491-1765925325621538", offset: "1_4", table: "child", where: "parent_id in (SELECT id FROM parent WHERE value = 1)", live: false}, 200, [%{"headers" => %{"event" => "move-out", "patterns" => [%{"pos" => 0, "value" => "ca55a23fed822f898710b30594b40dba"}]}}, %{"headers" => %{"control" => "snapshot-end", "xip_list" => [], "xmax" => "2393", "xmin" => "2393"}}, %{"headers" => %{"event" => "move-out", "patterns" => [%{"pos" => 0, "value" => "ca55a23fed822f898710b30594b40dba"}]}}, %{"headers" => %{"is_move_in" => true, "operation" => "insert", "relation" => ["public", "child"], "tags" => ["2c592528aae5a2772434233d4b846a41"]}, "key" => "\"public\".\"child\"/\"3\"", "value" => %{"id" => "3", "parent_id" => "3", "value" => "30"}}, %{"headers" => %{"control" => "snapshot-end", "xip_list" => [], "xmax" => "2393", "xmin" => "2393"}}, %{"headers" => %{"control" => "up-to-date", "global_last_seen_lsn" => "513843840"}}]}
     stacktrace:
       .../electric/plug/router_test.exs:2779: (test)
Elixir.Electric.Shapes.ConsumerTest::test event handling handles truncate when shape has a where clause
Stack Traces | 5.01s run time
4) test event handling handles truncate when shape has a where clause (Electric.Shapes.ConsumerTest)
     .../electric/shapes/consumer_test.exs:392
     Assertion failed, no matching message after 5000ms
     The following variables were pinned:
       pid = #PID<0.9169.0>
       ref = #Reference<0.2700037891.131072001.110430>
     Showing 8 of 8 messages in the mailbox
     code: assert_receive {:DOWN, ^ref, :process, ^pid, reason} when reason in [:shutdown, shutdown: :cleanup]
     mailbox:
       pattern: {:DOWN, ^ref, :process, ^pid, reason} when reason in [:shutdown, shutdown: :cleanup]
       value:   {Support.TestStorage, :for_shape, "Electric.Shapes.ConsumerTest-shape1"}

       pattern: {:DOWN, ^ref, :process, ^pid, reason} when reason in [:shutdown, shutdown: :cleanup]
       value:   {Support.TestStorage, :get_current_position, "Electric.Shapes.ConsumerTest-shape1"}

       pattern: {:DOWN, ^ref, :process, ^pid, reason} when reason in [:shutdown, shutdown: :cleanup]
       value:   {Support.TestStorage, :snapshot_started?, "Electric.Shapes.ConsumerTest-shape1"}

       pattern: {:DOWN, ^ref, :process, ^pid, reason} when reason in [:shutdown, shutdown: :cleanup]
       value:   {Support.TestStorage, :mark_snapshot_as_started, "Electric.Shapes.ConsumerTest-shape1"}

       pattern: {:DOWN, ^ref, :process, ^pid, reason} when reason in [:shutdown, shutdown: :cleanup]
       value:   {Support.TestStorage, :set_pg_snapshot, "Electric.Shapes.ConsumerTest-shape1", %{xmin: 100, xmax: 101, xip_list: ~c"d", filter_txns?: false}}

       pattern: {:DOWN, ^ref, :process, ^pid, reason} when reason in [:shutdown, shutdown: :cleanup]
       value:   {:DOWN, #Reference<0.2700037891.131072001.110430>, :process, #PID<0.9169.0>, :noproc}

       pattern: {:DOWN, ^ref, :process, ^pid, reason} when reason in [:shutdown, shutdown: :cleanup]
       value:   {Support.TestStorage, :cleanup!, "Electric.Shapes.ConsumerTest-shape1"}

       pattern: {:DOWN, ^ref, :process, ^pid, reason} when reason in [:shutdown, shutdown: :cleanup]
       value:   {Electric.ShapeCache.ShapeCleaner, :cleanup, "Electric.Shapes.ConsumerTest-shape1"}
     stacktrace:
       .../electric/shapes/consumer_test.exs:374: anonymous fn/3 in Electric.Shapes.ConsumerTest.assert_consumer_shutdown/4
       (elixir 1.19.1) lib/enum.ex:2520: Enum."-reduce/3-lists^foldl/2-0-"/3
       .../electric/shapes/consumer_test.exs:373: Electric.Shapes.ConsumerTest.assert_consumer_shutdown/4
       .../electric/shapes/consumer_test.exs:407: (test)

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@alco alco force-pushed the alco/ci-build-canary-images branch from c5fbc9d to cb04090 Compare December 16, 2025 22:38
@alco alco force-pushed the alco/ci-build-canary-images branch from cb04090 to f3d6cd7 Compare December 16, 2025 22:41
@alco alco marked this pull request as ready for review December 16, 2025 22:47
@alco alco changed the title Build Docker Hub images using GitHub Actions ci: Build Docker Hub images using GitHub Actions Dec 16, 2025
Copy link
Contributor

@robacourt robacourt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Great work!

@alco alco merged commit 7e2da0d into main Dec 17, 2025
47 of 49 checks passed
@alco alco deleted the alco/ci-build-canary-images branch December 17, 2025 13:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Retire Buildkite

3 participants