Skip to content

Fix concurrent map writes in pullRequiredImages function #12845

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 4 commits into from
Closed

Fix concurrent map writes in pullRequiredImages function #12845

wants to merge 4 commits into from

Conversation

0x2b3bfa0
Copy link

@0x2b3bfa0 0x2b3bfa0 commented May 17, 2025 •
edited
Loading

I found a panic in docker compose and asked GitHub Copilot to fix it. I don't have time to make myself responsible for the quality of this pull request, feel free to close it unmerged.

I wonder if the mutex should not be enclosed into the scope where it's being used, but should be passed from the scope that declares the images map:

compose/pkg/compose/build.go

Line 287 in 944e5e6

return s.pullRequiredImages(ctx, project, images, quietPull)

GitHub Copilot Description

Fix concurrent map writes error in pullRequiredImages function in pkg/compose/pull.go.

  • Add a sync.Mutex to protect concurrent access to the map images.
  • Lock the sync.Mutex before writing to the map images.
  • Unlock the sync.Mutex after writing to the map images.

For more details, open the Copilot Workspace session.

Panic Log

fatal error: concurrent map writes

goroutine 86 [running]:
github.com/docker/compose/v2/pkg/compose.(*composeService).pullRequiredImages.func1.1()
	github.com/docker/compose/v2/pkg/compose/pull.go:328 +0x236
golang.org/x/sync/errgroup.(*Group).Go.func1()
	golang.org/x/sync@v0.13.0/errgroup/errgroup.go:79 +0x50
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 84
	golang.org/x/sync@v0.13.0/errgroup/errgroup.go:76 +0x96

goroutine 1 [semacquire]:
sync.runtime_Semacquire(0xc00051aed0?)
	runtime/sema.go:71 +0x25
sync.(*WaitGroup).Wait(0x25c5480?)
	sync/waitgroup.go:118 +0x48
golang.org/x/sync/errgroup.(*Group).Wait(0xc0006a5040)
	golang.org/x/sync@v0.13.0/errgroup/errgroup.go:56 +0x25
github.com/docker/compose/v2/pkg/progress.RunWithStatus({0x2ba9388, 0xc0002db7d0}, 0xc00067fdf0, 0xc0003197d0, {0x27c8359, 0x7})
	github.com/docker/compose/v2/pkg/progress/writer.go:97 +0x225
github.com/docker/compose/v2/pkg/progress.Run({0x2ba9388, 0xc0002db7d0}, 0xc0006a5000, 0xc0003197d0)
	github.com/docker/compose/v2/pkg/progress/writer.go:61 +0x85
github.com/docker/compose/v2/pkg/compose.(*composeService).Up(0xc0003d64c0, {0x2ba9388, 0xc0002db7d0}, _, {{0x0, {0xc0003fe8a0, 0x0, 0x1}, 0x0, 0x0, ...}, ...})
	github.com/docker/compose/v2/pkg/compose/up.go:40 +0x213
github.com/docker/compose/v2/cmd/compose.runUp({_, _}, {_, _}, {_, _}, {0x0, 0x1, {0x27c670c, 0x6}, ...}, ...)
	github.com/docker/compose/v2/cmd/compose/up.go:319 +0xb54
github.com/docker/compose/v2/cmd/compose.upCommand.func2({0x2ba9388, 0xc0002db7d0}, 0xc0002c1290, {0xc0003fe8a0, 0x0, 0x1})
	github.com/docker/compose/v2/cmd/compose/up.go:143 +0x29f
github.com/docker/compose/v2/cmd/compose.upCommand.(*ProjectOptions).WithServices.func5({0x2ba93c0, 0xc00031a140}, {0xc0003fe8a0, 0x0, 0x1})
	github.com/docker/compose/v2/cmd/compose/compose.go:187 +0x22d
github.com/docker/compose/v2/cmd/compose.upCommand.(*ProjectOptions).WithServices.Adapt.func7({0x2ba93c0?, 0xc00031a140?}, 0x2?, {0xc0003fe8a0?, 0x2b8dfe8?, 0x4307a57?})
	github.com/docker/compose/v2/cmd/compose/compose.go:137 +0x30
github.com/docker/compose/v2/cmd/compose.upCommand.(*ProjectOptions).WithServices.Adapt.AdaptCmd.func8(0xc000596908, {0xc0003fe8a0, 0x0, 0x1})
	github.com/docker/compose/v2/cmd/compose/compose.go:121 +0x143
github.com/docker/cli/cli-plugins/plugin.RunPlugin.func1.1.2(0xc000596908, {0xc0003fe8a0, 0x0, 0x1})
	github.com/docker/cli@v28.1.0+incompatible/cli-plugins/plugin/plugin.go:65 +0x6c
github.com/docker/compose/v2/cmd/cmdtrace.Setup.wrapRunE.func2(0xc000596908?, {0xc0003fe8a0?, 0x0?, 0x1?})
	github.com/docker/compose/v2/cmd/cmdtrace/cmd_span.go:85 +0x63
github.com/spf13/cobra.(*Command).execute(0xc000596908, {0xc0003dcda0, 0x1, 0x1})
	github.com/spf13/cobra@v1.9.1/command.go:1015 +0xa94
github.com/spf13/cobra.(*Command).ExecuteC(0xc00049ef08)
	github.com/spf13/cobra@v1.9.1/command.go:1148 +0x40c
github.com/spf13/cobra.(*Command).Execute(...)
	github.com/spf13/cobra@v1.9.1/command.go:1071
github.com/docker/cli/cli-plugins/plugin.RunPlugin(0xc000242c80, 0xc000596608, {{0x27c5150, 0x5}, {0x27cf09e, 0xb}, {0x2b790c8, 0x7}, {0x0, 0x0}, ...})
	github.com/docker/cli@v28.1.0+incompatible/cli-plugins/plugin/plugin.go:80 +0x145
github.com/docker/cli/cli-plugins/plugin.Run(0x29213b0, {{0x27c5150, 0x5}, {0x27cf09e, 0xb}, {0x2b790c8, 0x7}, {0x0, 0x0}, {0x0, ...}})
	github.com/docker/cli@v28.1.0+incompatible/cli-plugins/plugin/plugin.go:95 +0x105
main.pluginMain()
	github.com/docker/compose/v2/cmd/main.go:38 +0xa5
main.main()
	github.com/docker/compose/v2/cmd/main.go:98 +0x19c

goroutine 35 [IO wait]:
internal/poll.runtime_pollWait(0x7f1ae48c66b0, 0x72)
	runtime/netpoll.go:351 +0x85
internal/poll.(*pollDesc).wait(0xc0004d3b00?, 0xc000086fbf?, 0x0)
	internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc0004d3b00, {0xc000086fbf, 0x1, 0x1})
	internal/poll/fd_unix.go:165 +0x27a
net.(*netFD).Read(0xc0004d3b00, {0xc000086fbf?, 0x0?, 0x0?})
	net/fd_posix.go:55 +0x25
net.(*conn).Read(0xc000125198, {0xc000086fbf?, 0x0?, 0x0?})
	net/net.go:189 +0x45
github.com/docker/cli/cli-plugins/socket.ConnectAndWait.func1()
	github.com/docker/cli@v28.1.0+incompatible/cli-plugins/socket/socket.go:162 +0x45
created by github.com/docker/cli/cli-plugins/socket.ConnectAndWait in goroutine 1
	github.com/docker/cli@v28.1.0+incompatible/cli-plugins/socket/socket.go:159 +0x118

goroutine 83 [select]:
github.com/docker/compose/v2/pkg/progress.(*plainWriter).Start(0xc00050a100, {0x2ba8f50, 0x412eac0})
	github.com/docker/compose/v2/pkg/progress/plain.go:34 +0x67
github.com/docker/compose/v2/pkg/progress.RunWithStatus.func1()
	github.com/docker/compose/v2/pkg/progress/writer.go:83 +0x2a
golang.org/x/sync/errgroup.(*Group).Go.func1()
	golang.org/x/sync@v0.13.0/errgroup/errgroup.go:79 +0x50
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 14
	golang.org/x/sync@v0.13.0/errgroup/errgroup.go:76 +0x96

goroutine 9 [select]:
go.opentelemetry.io/otel/sdk/trace.(*batchSpanProcessor).processQueue(0xc00058a000)
	go.opentelemetry.io/otel/sdk@v1.34.0/trace/batch_span_processor.go:302 +0x114
go.opentelemetry.io/otel/sdk/trace.NewBatchSpanProcessor.func1()
	go.opentelemetry.io/otel/sdk@v1.34.0/trace/batch_span_processor.go:117 +0x4e
created by go.opentelemetry.io/otel/sdk/trace.NewBatchSpanProcessor in goroutine 1
	go.opentelemetry.io/otel/sdk@v1.34.0/trace/batch_span_processor.go:115 +0x2e5

goroutine 49 [syscall]:
os/signal.signal_recv()
	runtime/sigqueue.go:152 +0x29
os/signal.loop()
	os/signal/signal_unix.go:23 +0x13
created by os/signal.Notify.func1.1 in goroutine 1
	os/signal/signal.go:151 +0x1f

goroutine 11 [chan receive]:
github.com/docker/compose/v2/cmd/compose.upCommand.AdaptCmd.func4.1()
	github.com/docker/compose/v2/cmd/compose/compose.go:115 +0x27
created by github.com/docker/compose/v2/cmd/compose.upCommand.AdaptCmd.func4 in goroutine 1
	github.com/docker/compose/v2/cmd/compose/compose.go:114 +0x10a

goroutine 12 [chan receive]:
github.com/docker/compose/v2/cmd/compose.upCommand.(*ProjectOptions).WithServices.Adapt.AdaptCmd.func8.1()
	github.com/docker/compose/v2/cmd/compose/compose.go:115 +0x27
created by github.com/docker/compose/v2/cmd/compose.upCommand.(*ProjectOptions).WithServices.Adapt.AdaptCmd.func8 in goroutine 1
	github.com/docker/compose/v2/cmd/compose/compose.go:114 +0x10a

goroutine 13 [select]:
github.com/docker/compose/v2/pkg/progress.(*plainWriter).Start(0xc00050afa0, {0x2ba8f50, 0x412eac0})
	github.com/docker/compose/v2/pkg/progress/plain.go:34 +0x67
github.com/docker/compose/v2/pkg/progress.RunWithStatus.func1()
	github.com/docker/compose/v2/pkg/progress/writer.go:83 +0x2a
golang.org/x/sync/errgroup.(*Group).Go.func1()
	golang.org/x/sync@v0.13.0/errgroup/errgroup.go:79 +0x50
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 1
	golang.org/x/sync@v0.13.0/errgroup/errgroup.go:76 +0x96

goroutine 14 [semacquire]:
sync.runtime_Semacquire(0xc00051a138?)
	runtime/sema.go:71 +0x25
sync.(*WaitGroup).Wait(0x25c5480?)
	sync/waitgroup.go:118 +0x48
golang.org/x/sync/errgroup.(*Group).Wait(0xc0006a45c0)
	golang.org/x/sync@v0.13.0/errgroup/errgroup.go:56 +0x25
github.com/docker/compose/v2/pkg/progress.RunWithStatus({0x2ba9388, 0xc000705950}, 0xc0006a3800, 0xc0003197d0, {0x27c8359, 0x7})
	github.com/docker/compose/v2/pkg/progress/writer.go:97 +0x225
github.com/docker/compose/v2/pkg/progress.Run({0x2ba9388, 0xc000705950}, 0xc0007059b0, 0xc0003197d0)
	github.com/docker/compose/v2/pkg/progress/writer.go:61 +0x85
github.com/docker/compose/v2/pkg/compose.(*composeService).pullRequiredImages(0xc0003d64c0, {0x2ba9388, 0xc000705950}, 0xc0002c1290, 0xc00068ee70, 0x0)
	github.com/docker/compose/v2/pkg/compose/pull.go:320 +0x2ef
github.com/docker/compose/v2/pkg/compose.(*composeService).ensureImagesExists.func1({0x2ba9388?, 0xc000705950?})
	github.com/docker/compose/v2/pkg/compose/build.go:278 +0x37
github.com/docker/compose/v2/pkg/compose.(*composeService).ensureImagesExists.SpanWrapFunc.func3({0x2ba9388, 0xc00068ee40})
	github.com/docker/compose/v2/internal/tracing/wrap.go:43 +0x13d
github.com/docker/compose/v2/pkg/compose.(*composeService).ensureImagesExists(0xc0003d64c0, {0x2ba9388, 0xc00068ee40}, 0xc0002c1290, 0x0, 0x0)
	github.com/docker/compose/v2/pkg/compose/build.go:280 +0x29e
github.com/docker/compose/v2/pkg/compose.(*composeService).create(0xc0003d64c0, {0x2ba9388, 0xc00068ee40}, 0xc0002c1290, {0x0, {0xc000508400, 0xe, 0x10}, 0x0, 0x0, ...})
	github.com/docker/compose/v2/pkg/compose/create.go:83 +0xdf
github.com/docker/compose/v2/pkg/compose.(*composeService).Up.func1({0x2ba9388, 0xc00068ee40})
	github.com/docker/compose/v2/pkg/compose/up.go:41 +0x85
github.com/docker/compose/v2/pkg/compose.(*composeService).Up.SpanWrapFunc.func5({0x2ba9388, 0xc00068ee10})
	github.com/docker/compose/v2/internal/tracing/wrap.go:43 +0x13d
github.com/docker/compose/v2/pkg/progress.Run.func1({0x2ba9388?, 0xc00068ee10?})
	github.com/docker/compose/v2/pkg/progress/writer.go:62 +0x22
github.com/docker/compose/v2/pkg/progress.RunWithStatus.func2()
	github.com/docker/compose/v2/pkg/progress/writer.go:90 +0x70
golang.org/x/sync/errgroup.(*Group).Go.func1()
	golang.org/x/sync@v0.13.0/errgroup/errgroup.go:79 +0x50
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 1
	golang.org/x/sync@v0.13.0/errgroup/errgroup.go:76 +0x96

goroutine 23 [select]:
net/http.(*persistConn).readLoop(0xc0007426c0)
	net/http/transport.go:2325 +0xca5
created by net/http.(*Transport).dialConn in goroutine 22
	net/http/transport.go:1874 +0x154f

goroutine 79 [select]:
net/http.(*persistConn).readLoop(0xc0007987e0)
	net/http/transport.go:2325 +0xca5
created by net/http.(*Transport).dialConn in goroutine 78
	net/http/transport.go:1874 +0x154f

goroutine 54 [select]:
net/http.(*persistConn).readLoop(0xc00023b9e0)
	net/http/transport.go:2325 +0xca5
created by net/http.(*Transport).dialConn in goroutine 53
	net/http/transport.go:1874 +0x154f

goroutine 80 [select]:
net/http.(*persistConn).writeLoop(0xc0007987e0)
	net/http/transport.go:2519 +0xe7
created by net/http.(*Transport).dialConn in goroutine 78
	net/http/transport.go:1875 +0x15a5

goroutine 20 [IO wait]:
internal/poll.runtime_pollWait(0x7f1ae48c6250, 0x72)
	runtime/netpoll.go:351 +0x85
internal/poll.(*pollDesc).wait(0xc0001a2000?, 0xc000836000?, 0x0)
	internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc0001a2000, {0xc000836000, 0x1000, 0x1000})
	internal/poll/fd_unix.go:165 +0x27a
net.(*netFD).Read(0xc0001a2000, {0xc000836000?, 0x0?, 0x2b805a0?})
	net/fd_posix.go:55 +0x25
net.(*conn).Read(0xc0004b8010, {0xc000836000?, 0x0?, 0x0?})
	net/net.go:189 +0x45
net/http.(*persistConn).Read(0xc00082e000, {0xc000836000?, 0x777f25?, 0x2379d40?})
	net/http/transport.go:2052 +0x4a
bufio.(*Reader).fill(0xc00053a060)
	bufio/bufio.go:110 +0x103
bufio.(*Reader).Peek(0xc00053a060, 0x1)
	bufio/bufio.go:148 +0x53
net/http.(*persistConn).readLoop(0xc00082e000)
	net/http/transport.go:2205 +0x185
created by net/http.(*Transport).dialConn in goroutine 19
	net/http/transport.go:1874 +0x154f

goroutine 43 [select]:
net/http.(*persistConn).readLoop(0xc000742120)
	net/http/transport.go:2325 +0xca5
created by net/http.(*Transport).dialConn in goroutine 42
	net/http/transport.go:1874 +0x154f

goroutine 44 [select]:
net/http.(*persistConn).writeLoop(0xc000742120)
	net/http/transport.go:2519 +0xe7
created by net/http.(*Transport).dialConn in goroutine 42
	net/http/transport.go:1875 +0x15a5

goroutine 55 [select]:
net/http.(*persistConn).writeLoop(0xc00023b9e0)
	net/http/transport.go:2519 +0xe7
created by net/http.(*Transport).dialConn in goroutine 53
	net/http/transport.go:1875 +0x15a5

goroutine 24 [select]:
net/http.(*persistConn).writeLoop(0xc0007426c0)
	net/http/transport.go:2519 +0xe7
created by net/http.(*Transport).dialConn in goroutine 22
	net/http/transport.go:1875 +0x15a5

goroutine 21 [select]:
net/http.(*persistConn).writeLoop(0xc00082e000)
	net/http/transport.go:2519 +0xe7
created by net/http.(*Transport).dialConn in goroutine 19
	net/http/transport.go:1875 +0x15a5

goroutine 84 [semacquire]:
sync.runtime_Semacquire(0x0?)
	runtime/sema.go:71 +0x25
sync.(*WaitGroup).Wait(0xc000705980?)
	sync/waitgroup.go:118 +0x48
golang.org/x/sync/errgroup.(*Group).Wait(0xc0006a4700)
	golang.org/x/sync@v0.13.0/errgroup/errgroup.go:56 +0x25
github.com/docker/compose/v2/pkg/compose.(*composeService).pullRequiredImages.func1({0x2ba9388, 0xc0007059e0})
	github.com/docker/compose/v2/pkg/compose/pull.go:340 +0x39b
github.com/docker/compose/v2/pkg/progress.Run.func1({0x2ba9388?, 0xc0007059e0?})
	github.com/docker/compose/v2/pkg/progress/writer.go:62 +0x22
github.com/docker/compose/v2/pkg/progress.RunWithStatus.func2()
	github.com/docker/compose/v2/pkg/progress/writer.go:90 +0x70
golang.org/x/sync/errgroup.(*Group).Go.func1()
	golang.org/x/sync@v0.13.0/errgroup/errgroup.go:79 +0x50
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 14
	golang.org/x/sync@v0.13.0/errgroup/errgroup.go:76 +0x96

goroutine 87 [IO wait]:
internal/poll.runtime_pollWait(0x7f1ae48c6020, 0x72)
	runtime/netpoll.go:351 +0x85
internal/poll.(*pollDesc).wait(0xc0002ba200?, 0xc00074f000?, 0x0)
	internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc0002ba200, {0xc00074f000, 0x1000, 0x1000})
	internal/poll/fd_unix.go:165 +0x27a
net.(*netFD).Read(0xc0002ba200, {0xc00074f000?, 0x0?, 0x0?})
	net/fd_posix.go:55 +0x25
net.(*conn).Read(0xc000530088, {0xc00074f000?, 0x0?, 0xc00089ea80?})
	net/net.go:189 +0x45
net/http.(*persistConn).Read(0xc000742120, {0xc00074f000?, 0x0?, 0x60?})
	net/http/transport.go:2052 +0x4a
bufio.(*Reader).fill(0xc00021d2c0)
	bufio/bufio.go:110 +0x103
bufio.(*Reader).ReadSlice(0xc00021d2c0, 0xa)
	bufio/bufio.go:376 +0x29
net/http/internal.readChunkLine(0x0?)
	net/http/internal/chunked.go:156 +0x1c
net/http/internal.(*chunkedReader).beginChunk(0xc00083c210)
	net/http/internal/chunked.go:49 +0x25
net/http/internal.(*chunkedReader).Read(0xc00083c210, {0xc0001c9202?, 0x0?, 0xc0008f0650?})
	net/http/internal/chunked.go:125 +0x131
net/http.(*body).readLocked(0xc0006a4a80, {0xc0001c9202?, 0xc00083f450?, 0xc0002b0080?})
	net/http/transfer.go:844 +0x3b
net/http.(*body).Read(0x4?, {0xc0001c9202?, 0x223ed20?, 0x4132600?})
	net/http/transfer.go:836 +0x112
net/http.(*bodyEOFSignal).Read(0xc0006a4ac0, {0xc0001c9202, 0x5fe, 0x5fe})
	net/http/transport.go:2913 +0x13f
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*wrappedBody).Read(0xc0006a4b80, {0xc0001c9202?, 0x0?, 0x0?})
	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.56.0/transport.go:229 +0x2d
encoding/json.(*Decoder).refill(0xc000388b40)
	encoding/json/stream.go:165 +0x188
encoding/json.(*Decoder).readValue(0xc000388b40)
	encoding/json/stream.go:140 +0x85
encoding/json.(*Decoder).Decode(0xc000388b40, {0x235b6a0, 0xc00081b050})
	encoding/json/stream.go:63 +0x75
github.com/docker/compose/v2/pkg/compose.(*composeService).pullServiceImage(_, {_, _}, {{0xc0004a64f8, 0x5}, {0x0, 0x0, 0x0}, 0x0, 0x0, ...}, ...)
	github.com/docker/compose/v2/pkg/compose/pull.go:231 +0x4b0
github.com/docker/compose/v2/pkg/compose.(*composeService).pullRequiredImages.func1.1()
	github.com/docker/compose/v2/pkg/compose/pull.go:327 +0x170
golang.org/x/sync/errgroup.(*Group).Go.func1()
	golang.org/x/sync@v0.13.0/errgroup/errgroup.go:79 +0x50
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 84
	golang.org/x/sync@v0.13.0/errgroup/errgroup.go:76 +0x96

goroutine 88 [IO wait]:
internal/poll.runtime_pollWait(0x7f1ae48c5df0, 0x72)
	runtime/netpoll.go:351 +0x85
internal/poll.(*pollDesc).wait(0xc0003b2a80?, 0xc000920000?, 0x0)
	internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc0003b2a80, {0xc000920000, 0x1000, 0x1000})
	internal/poll/fd_unix.go:165 +0x27a
net.(*netFD).Read(0xc0003b2a80, {0xc000920000?, 0x0?, 0x0?})
	net/fd_posix.go:55 +0x25
net.(*conn).Read(0xc0004b8160, {0xc000920000?, 0x0?, 0xc00089ec40?})
	net/net.go:189 +0x45
net/http.(*persistConn).Read(0xc00082efc0, {0xc000920000?, 0x0?, 0x17?})
	net/http/transport.go:2052 +0x4a
bufio.(*Reader).fill(0xc000676660)
	bufio/bufio.go:110 +0x103
bufio.(*Reader).ReadSlice(0xc000676660, 0xa)
	bufio/bufio.go:376 +0x29
net/http/internal.readChunkLine(0x0?)
	net/http/internal/chunked.go:156 +0x1c
net/http/internal.(*chunkedReader).beginChunk(0xc00083c450)
	net/http/internal/chunked.go:49 +0x25
net/http/internal.(*chunkedReader).Read(0xc00083c450, {0xc000922c02?, 0x0?, 0xc00094e650?})
	net/http/internal/chunked.go:125 +0x131
net/http.(*body).readLocked(0xc0006a5380, {0xc000922c02?, 0xc00087c680?, 0xc00081c3c0?})
	net/http/transfer.go:844 +0x3b
net/http.(*body).Read(0x4?, {0xc000922c02?, 0x223ed20?, 0x4132600?})
	net/http/transfer.go:836 +0x112
net/http.(*bodyEOFSignal).Read(0xc0006a53c0, {0xc000922c02, 0x5fe, 0x5fe})
	net/http/transport.go:2913 +0x13f
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*wrappedBody).Read(0xc0006a5b40, {0xc000922c02?, 0x0?, 0x0?})
	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.56.0/transport.go:229 +0x2d
encoding/json.(*Decoder).refill(0xc000389540)
	encoding/json/stream.go:165 +0x188
encoding/json.(*Decoder).readValue(0xc000389540)
	encoding/json/stream.go:140 +0x85
encoding/json.(*Decoder).Decode(0xc000389540, {0x235b6a0, 0xc00031f170})
	encoding/json/stream.go:63 +0x75
github.com/docker/compose/v2/pkg/compose.(*composeService).pullServiceImage(_, {_, _}, {{0xc000539fe0, 0x8}, {0x0, 0x0, 0x0}, 0x0, 0x0, ...}, ...)
	github.com/docker/compose/v2/pkg/compose/pull.go:231 +0x4b0
github.com/docker/compose/v2/pkg/compose.(*composeService).pullRequiredImages.func1.1()
	github.com/docker/compose/v2/pkg/compose/pull.go:327 +0x170
golang.org/x/sync/errgroup.(*Group).Go.func1()
	golang.org/x/sync@v0.13.0/errgroup/errgroup.go:79 +0x50
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 84
	golang.org/x/sync@v0.13.0/errgroup/errgroup.go:76 +0x96

goroutine 89 [IO wait]:
internal/poll.runtime_pollWait(0x7f1ae48c5f08, 0x72)
	runtime/netpoll.go:351 +0x85
internal/poll.(*pollDesc).wait(0xc0002ba100?, 0xc00076f000?, 0x0)
	internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc0002ba100, {0xc00076f000, 0x1000, 0x1000})
	internal/poll/fd_unix.go:165 +0x27a
net.(*netFD).Read(0xc0002ba100, {0xc00076f000?, 0x0?, 0x0?})
	net/fd_posix.go:55 +0x25
net.(*conn).Read(0xc000124058, {0xc00076f000?, 0x0?, 0xc00089ee00?})
	net/net.go:189 +0x45
net/http.(*persistConn).Read(0xc0007987e0, {0xc00076f000?, 0x0?, 0x21?})
	net/http/transport.go:2052 +0x4a
bufio.(*Reader).fill(0xc00053a480)
	bufio/bufio.go:110 +0x103
bufio.(*Reader).ReadSlice(0xc00053a480, 0xa)
	bufio/bufio.go:376 +0x29
net/http/internal.readChunkLine(0x0?)
	net/http/internal/chunked.go:156 +0x1c
net/http/internal.(*chunkedReader).beginChunk(0xc00083c3c0)
	net/http/internal/chunked.go:49 +0x25
net/http/internal.(*chunkedReader).Read(0xc00083c3c0, {0xc000922602?, 0x0?, 0xc000480650?})
	net/http/internal/chunked.go:125 +0x131
net/http.(*body).readLocked(0xc0006a5240, {0xc000922602?, 0xc00083f450?, 0xc0002b0080?})
	net/http/transfer.go:844 +0x3b
net/http.(*body).Read(0x4?, {0xc000922602?, 0x223ed20?, 0x4132600?})
	net/http/transfer.go:836 +0x112
net/http.(*bodyEOFSignal).Read(0xc0006a5280, {0xc000922602, 0x5fe, 0x5fe})
	net/http/transport.go:2913 +0x13f
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*wrappedBody).Read(0xc0006a5340, {0xc000922602?, 0x0?, 0x0?})
	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.56.0/transport.go:229 +0x2d
encoding/json.(*Decoder).refill(0xc000389400)
	encoding/json/stream.go:165 +0x188
encoding/json.(*Decoder).readValue(0xc000389400)
	encoding/json/stream.go:140 +0x85
encoding/json.(*Decoder).Decode(0xc000389400, {0x235b6a0, 0xc00081b170})
	encoding/json/stream.go:63 +0x75
github.com/docker/compose/v2/pkg/compose.(*composeService).pullServiceImage(_, {_, _}, {{0xc0004a6180, 0xa}, {0x0, 0x0, 0x0}, 0x0, 0x0, ...}, ...)
	github.com/docker/compose/v2/pkg/compose/pull.go:231 +0x4b0
github.com/docker/compose/v2/pkg/compose.(*composeService).pullRequiredImages.func1.1()
	github.com/docker/compose/v2/pkg/compose/pull.go:327 +0x170
golang.org/x/sync/errgroup.(*Group).Go.func1()
	golang.org/x/sync@v0.13.0/errgroup/errgroup.go:79 +0x50
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 84
	golang.org/x/sync@v0.13.0/errgroup/errgroup.go:76 +0x96

goroutine 90 [IO wait]:
internal/poll.runtime_pollWait(0x7f1ae48c6598, 0x72)
	runtime/netpoll.go:351 +0x85
internal/poll.(*pollDesc).wait(0xc0003fc080?, 0xc000142000?, 0x0)
	internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc0003fc080, {0xc000142000, 0x1000, 0x1000})
	internal/poll/fd_unix.go:165 +0x27a
net.(*netFD).Read(0xc0003fc080, {0xc000142000?, 0x0?, 0x0?})
	net/fd_posix.go:55 +0x25
net.(*conn).Read(0xc000530048, {0xc000142000?, 0x0?, 0xc00089efc0?})
	net/net.go:189 +0x45
net/http.(*persistConn).Read(0xc0007426c0, {0xc000142000?, 0x0?, 0x17?})
	net/http/transport.go:2052 +0x4a
bufio.(*Reader).fill(0xc00021daa0)
	bufio/bufio.go:110 +0x103
bufio.(*Reader).ReadSlice(0xc00021daa0, 0xa)
	bufio/bufio.go:376 +0x29
net/http/internal.readChunkLine(0x0?)
	net/http/internal/chunked.go:156 +0x1c
net/http/internal.(*chunkedReader).beginChunk(0xc00083c4e0)
	net/http/internal/chunked.go:49 +0x25
net/http/internal.(*chunkedReader).Read(0xc00083c4e0, {0xc000923202?, 0x0?, 0xc00080a650?})
	net/http/internal/chunked.go:125 +0x131
net/http.(*body).readLocked(0xc0006a5b80, {0xc000923202?, 0xc00087c680?, 0xc00081c3c0?})
	net/http/transfer.go:844 +0x3b
net/http.(*body).Read(0x4?, {0xc000923202?, 0x223ed20?, 0x4132600?})
	net/http/transfer.go:836 +0x112
net/http.(*bodyEOFSignal).Read(0xc0006a5bc0, {0xc000923202, 0x5fe, 0x5fe})
	net/http/transport.go:2913 +0x13f
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*wrappedBody).Read(0xc0006a5c80, {0xc000923202?, 0x0?, 0x0?})
	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.56.0/transport.go:229 +0x2d
encoding/json.(*Decoder).refill(0xc000389680)
	encoding/json/stream.go:165 +0x188
encoding/json.(*Decoder).readValue(0xc000389680)
	encoding/json/stream.go:140 +0x85
encoding/json.(*Decoder).Decode(0xc000389680, {0x235b6a0, 0xc00031e870})
	encoding/json/stream.go:63 +0x75
github.com/docker/compose/v2/pkg/compose.(*composeService).pullServiceImage(_, {_, _}, {{0xc0004a62d0, 0x5}, {0x0, 0x0, 0x0}, 0x0, 0x0, ...}, ...)
	github.com/docker/compose/v2/pkg/compose/pull.go:231 +0x4b0
github.com/docker/compose/v2/pkg/compose.(*composeService).pullRequiredImages.func1.1()
	github.com/docker/compose/v2/pkg/compose/pull.go:327 +0x170
golang.org/x/sync/errgroup.(*Group).Go.func1()
	golang.org/x/sync@v0.13.0/errgroup/errgroup.go:79 +0x50
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 84
	golang.org/x/sync@v0.13.0/errgroup/errgroup.go:76 +0x96

goroutine 91 [IO wait]:
internal/poll.runtime_pollWait(0x7f1ae48c6138, 0x72)
	runtime/netpoll.go:351 +0x85
internal/poll.(*pollDesc).wait(0xc0001a2100?, 0xc0002f4000?, 0x0)
	internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc0001a2100, {0xc0002f4000, 0x1000, 0x1000})
	internal/poll/fd_unix.go:165 +0x27a
net.(*netFD).Read(0xc0001a2100, {0xc0002f4000?, 0x0?, 0x0?})
	net/fd_posix.go:55 +0x25
net.(*conn).Read(0xc00008e308, {0xc0002f4000?, 0x0?, 0xc00089f180?})
	net/net.go:189 +0x45
net/http.(*persistConn).Read(0xc00023b9e0, {0xc0002f4000?, 0x0?, 0x17?})
	net/http/transport.go:2052 +0x4a
bufio.(*Reader).fill(0xc000119320)
	bufio/bufio.go:110 +0x103
bufio.(*Reader).ReadSlice(0xc000119320, 0xa)
	bufio/bufio.go:376 +0x29
net/http/internal.readChunkLine(0x0?)
	net/http/internal/chunked.go:156 +0x1c
net/http/internal.(*chunkedReader).beginChunk(0xc00083c570)
	net/http/internal/chunked.go:49 +0x25
net/http/internal.(*chunkedReader).Read(0xc00083c570, {0xc000923802?, 0x0?, 0xc00047c650?})
	net/http/internal/chunked.go:125 +0x131
net/http.(*body).readLocked(0xc0006a5cc0, {0xc000923802?, 0xc00087c680?, 0xc00081c3c0?})
	net/http/transfer.go:844 +0x3b
net/http.(*body).Read(0x4?, {0xc000923802?, 0x223ed20?, 0x4132600?})
	net/http/transfer.go:836 +0x112
net/http.(*bodyEOFSignal).Read(0xc0006a5d00, {0xc000923802, 0x5fe, 0x5fe})
	net/http/transport.go:2913 +0x13f
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*wrappedBody).Read(0xc0006a5dc0, {0xc000923802?, 0x0?, 0x0?})
	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.56.0/transport.go:229 +0x2d
encoding/json.(*Decoder).refill(0xc0003897c0)
	encoding/json/stream.go:165 +0x188
encoding/json.(*Decoder).readValue(0xc0003897c0)
	encoding/json/stream.go:140 +0x85
encoding/json.(*Decoder).Decode(0xc0003897c0, {0x235b6a0, 0xc00031fa70})
	encoding/json/stream.go:63 +0x75
github.com/docker/compose/v2/pkg/compose.(*composeService).pullServiceImage(_, {_, _}, {{0xc0004a6f80, 0x10}, {0x0, 0x0, 0x0}, 0x0, 0x0, ...}, ...)
	github.com/docker/compose/v2/pkg/compose/pull.go:231 +0x4b0
github.com/docker/compose/v2/pkg/compose.(*composeService).pullRequiredImages.func1.1()
	github.com/docker/compose/v2/pkg/compose/pull.go:327 +0x170
golang.org/x/sync/errgroup.(*Group).Go.func1()
	golang.org/x/sync@v0.13.0/errgroup/errgroup.go:79 +0x50
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 84
	golang.org/x/sync@v0.13.0/errgroup/errgroup.go:76 +0x96

goroutine 92 [IO wait]:
internal/poll.runtime_pollWait(0x7f1ae48c5cd8, 0x72)
	runtime/netpoll.go:351 +0x85
internal/poll.(*pollDesc).wait(0xc0002ba480?, 0xc0008f2000?, 0x0)
	internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc0002ba480, {0xc0008f2000, 0x1000, 0x1000})
	internal/poll/fd_unix.go:165 +0x27a
net.(*netFD).Read(0xc0002ba480, {0xc0008f2000?, 0x0?, 0x0?})
	net/fd_posix.go:55 +0x25
net.(*conn).Read(0xc0001240b8, {0xc0008f2000?, 0x0?, 0xc00089f340?})
	net/net.go:189 +0x45
net/http.(*persistConn).Read(0xc000798900, {0xc0008f2000?, 0x0?, 0x17?})
	net/http/transport.go:2052 +0x4a
bufio.(*Reader).fill(0xc00053a6c0)
	bufio/bufio.go:110 +0x103
bufio.(*Reader).ReadSlice(0xc00053a6c0, 0xa)
	bufio/bufio.go:376 +0x29
net/http/internal.readChunkLine(0x0?)
	net/http/internal/chunked.go:156 +0x1c
net/http/internal.(*chunkedReader).beginChunk(0xc00083c330)
	net/http/internal/chunked.go:49 +0x25
net/http/internal.(*chunkedReader).Read(0xc00083c330, {0xc000922002?, 0x0?, 0xc00094a650?})
	net/http/internal/chunked.go:125 +0x131
net/http.(*body).readLocked(0xc0006a50c0, {0xc000922002?, 0xc00087c680?, 0xc00081c3c0?})
	net/http/transfer.go:844 +0x3b
net/http.(*body).Read(0x4?, {0xc000922002?, 0x223ed20?, 0x4132600?})
	net/http/transfer.go:836 +0x112
net/http.(*bodyEOFSignal).Read(0xc0006a5140, {0xc000922002, 0x5fe, 0x5fe})
	net/http/transport.go:2913 +0x13f
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*wrappedBody).Read(0xc0006a5200, {0xc000922002?, 0x0?, 0x0?})
	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.56.0/transport.go:229 +0x2d
encoding/json.(*Decoder).refill(0xc0003892c0)
	encoding/json/stream.go:165 +0x188
encoding/json.(*Decoder).readValue(0xc0003892c0)
	encoding/json/stream.go:140 +0x85
encoding/json.(*Decoder).Decode(0xc0003892c0, {0x235b6a0, 0xc00031e7e0})
	encoding/json/stream.go:63 +0x75
github.com/docker/compose/v2/pkg/compose.(*composeService).pullServiceImage(_, {_, _}, {{0xc0004a6919, 0x5}, {0x0, 0x0, 0x0}, 0x0, 0x0, ...}, ...)
	github.com/docker/compose/v2/pkg/compose/pull.go:231 +0x4b0
github.com/docker/compose/v2/pkg/compose.(*composeService).pullRequiredImages.func1.1()
	github.com/docker/compose/v2/pkg/compose/pull.go:327 +0x170
golang.org/x/sync/errgroup.(*Group).Go.func1()
	golang.org/x/sync@v0.13.0/errgroup/errgroup.go:79 +0x50
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 84
	golang.org/x/sync@v0.13.0/errgroup/errgroup.go:76 +0x96

goroutine 94 [select]:
net/http.(*persistConn).readLoop(0xc00082efc0)
	net/http/transport.go:2325 +0xca5
created by net/http.(*Transport).dialConn in goroutine 93
	net/http/transport.go:1874 +0x154f

goroutine 95 [select]:
net/http.(*persistConn).writeLoop(0xc00082efc0)
	net/http/transport.go:2519 +0xe7
created by net/http.(*Transport).dialConn in goroutine 93
	net/http/transport.go:1875 +0x15a5

goroutine 98 [select]:
net/http.(*persistConn).readLoop(0xc000798900)
	net/http/transport.go:2325 +0xca5
created by net/http.(*Transport).dialConn in goroutine 97
	net/http/transport.go:1874 +0x154f

goroutine 99 [select]:
net/http.(*persistConn).writeLoop(0xc000798900)
	net/http/transport.go:2519 +0xe7
created by net/http.(*Transport).dialConn in goroutine 97
	net/http/transport.go:1875 +0x15a5

goroutine 101 [IO wait]:
internal/poll.runtime_pollWait(0x7f1ae48c6368, 0x72)
	runtime/netpoll.go:351 +0x85
internal/poll.(*pollDesc).wait(0xc0002ba700?, 0xc0008fe000?, 0x0)
	internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc0002ba700, {0xc0008fe000, 0x1000, 0x1000})
	internal/poll/fd_unix.go:165 +0x27a
net.(*netFD).Read(0xc0002ba700, {0xc0008fe000?, 0x0?, 0x2b805a0?})
	net/fd_posix.go:55 +0x25
net.(*conn).Read(0xc000124118, {0xc0008fe000?, 0x0?, 0x0?})
	net/net.go:189 +0x45
net/http.(*persistConn).Read(0xc000798a20, {0xc0008fe000?, 0x777f25?, 0x2379d40?})
	net/http/transport.go:2052 +0x4a
bufio.(*Reader).fill(0xc00053a900)
	bufio/bufio.go:110 +0x103
bufio.(*Reader).Peek(0xc00053a900, 0x1)
	bufio/bufio.go:148 +0x53
net/http.(*persistConn).readLoop(0xc000798a20)
	net/http/transport.go:2205 +0x185
created by net/http.(*Transport).dialConn in goroutine 100
	net/http/transport.go:1874 +0x154f

goroutine 102 [select]:
net/http.(*persistConn).writeLoop(0xc000798a20)
	net/http/transport.go:2519 +0xe7
created by net/http.(*Transport).dialConn in goroutine 100
	net/http/transport.go:1875 +0x15a5

@0x2b3bfa0
Fix concurrent map writes in pullRequiredImages function
72e2a6a 
Fix concurrent map writes error in `pullRequiredImages` function in `pkg/compose/pull.go`.

* Add a `sync.Mutex` to protect concurrent access to the map `images`.
* Lock the `sync.Mutex` before writing to the map `images`.
* Unlock the `sync.Mutex` after writing to the map `images`.

---

For more details, open the [Copilot Workspace session](https://copilot-workspace.githubnext.com/docker/compose?shareId=XXXX-XXXX-XXXX-XXXX).
@0x2b3bfa0 0x2b3bfa0 requested a review from a team as a code owner May 17, 2025 13:21
@0x2b3bfa0 0x2b3bfa0 requested review from ndeloof and glours May 17, 2025 13:21
0x2b3bfa0 added 3 commits May 17, 2025 15:22
@0x2b3bfa0
Update pull.go
b1f1fb5 
Signed-off-by: Helio Machado <0x2b3bfa0+git@googlemail.com>
@0x2b3bfa0
Add mutexes to protect concurrent access to maps in `pullRequiredImag…
70eaaf4 
…es` function

* Add `imagesMutex` to protect concurrent access to the `images` map
* Add `pulledImagesMutex` to protect concurrent access to the `pulledImages` map
* Lock and unlock `pulledImagesMutex` before and after writing to the `pulledImages` map
* Lock and unlock `imagesMutex` before and after reading from the `images` map
@0x2b3bfa0
Update pull.go
4c555ce 
Signed-off-by: Helio Machado <0x2b3bfa0+git@googlemail.com>
ndeloof
ndeloof reviewed May 17, 2025
View reviewed changes
Copy link
Contributor

@ndeloof ndeloof left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't seem relevant to me
concurrent writes to pulledImages map are already guarded by a mutex
later access takes place after eg.Wait() so doesn't required synchronization

@ndeloof
Copy link
Contributor

ndeloof commented May 17, 2025

already fixed by #12752

@ndeloof ndeloof closed this May 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ndeloof ndeloof ndeloof left review comments

@glours glours Awaiting requested review from glours glours is a code owner automatically assigned from docker/compose-maintainers

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@0x2b3bfa0 @ndeloof