Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Jan 26, 2026

The spinner could deadlock when Stop() or StopWithMessage() was called before or immediately after Start() launched its goroutine. This manifested intermittently in gh aw run when workflow runs appeared quickly.

Root Cause

Start() launches program.Run() asynchronously. If Stop() calls program.Quit() before the goroutine initializes bubbletea's internal channels, it blocks forever waiting for a receiver that doesn't exist yet.

// BEFORE: Race condition
func (s *SpinnerWrapper) Start() {
    if s.enabled && s.program != nil {
        go func() { _, _ = s.program.Run() }()  // Async initialization
    }
}

func (s *SpinnerWrapper) Stop() {
    if s.enabled && s.program != nil {
        s.program.Quit()  // May block if goroutine hasn't started
    }
}

Changes

Added lifecycle tracking:

  • running flag and sync.Mutex to track whether program.Run() is active
  • Start() sets running=true before launching goroutine
  • Stop()/StopWithMessage() only call program.Quit() when running==true
  • UpdateMessage() only sends when running==true

Preserved user feedback:

  • StopWithMessage() prints its message even when spinner wasn't started, ensuring users always see status updates
// AFTER: Safe lifecycle management
func (s *SpinnerWrapper) Stop() {
    if s.enabled && s.program != nil {
        s.mu.Lock()
        if s.running {
            s.running = false
            s.mu.Unlock()
            s.program.Quit()  // Only called when safe
        } else {
            s.mu.Unlock()
        }
    }
}

Test Coverage

Added TestSpinnerStopBeforeStartRaceCondition that exercises 100 rapid Start/Stop cycles to catch this race condition.

Original prompt

This section details on the original issue you should resolve

<issue_title>fatal error: all goroutines are asleep - deadlock! goroutine 1</issue_title>
<issue_description>gh aw run daily-team-status
yields sometimes the following error:

it seems to be a timing issue or releated to performance of task.

Most of the time it disappears if i run it with --verbose flag
gh aw run daily-team-status --verbose

appened is the stacktrace of:
gh aw run daily-team-status

Successfully triggered workflow: daily-team-status.lock.yml fatal error: all goroutines are asleep - deadlock! goroutine 1 [select]: github.com/charmbracelet/bubbletea.(*Program).Send(...) github.com/charmbracelet/bubbletea@v1.3.10/tea.go:775 github.com/charmbracelet/bubbletea.(*Program).Quit(...) github.com/charmbracelet/bubbletea@v1.3.10/tea.go:789 github.com/githubnext/gh-aw/pkg/console.(*SpinnerWrapper).StopWithMessage(0x0?, {0xf667f8, 0x16}) github.com/githubnext/gh-aw/pkg/console/spinner.go:113 +0xbd github.com/githubnext/gh-aw/pkg/cli.getLatestWorkflowRunWithRetry({0xc0003d2040, 0x1a}, {0x0, 0x0}, 0x0) github.com/githubnext/gh-aw/pkg/cli/run_workflow_tracking.go:152 +0x12f9 github.com/githubnext/gh-aw/pkg/cli.RunWorkflowOnGitHub({0x122afb8, 0x18838c0}, {0x7fff184085af, 0x11}, 0x0, {0x0, 0x0}, {0x0, 0x0}, {0x0, ...}, ...) github.com/githubnext/gh-aw/pkg/cli/run_workflow_execution.go:455 +0x3ac6 github.com/githubnext/gh-aw/pkg/cli.RunWorkflowsOnGitHub.func1() github.com/githubnext/gh-aw/pkg/cli/run_workflow_execution.go:588 +0x4cf github.com/githubnext/gh-aw/pkg/cli.ExecuteWithRepeat({0x0, {0x0, 0x0}, {0xf667e2, 0x16}, 0xc000213880, 0x0, 0x0}) github.com/githubnext/gh-aw/pkg/cli/retry.go:38 +0xbf github.com/githubnext/gh-aw/pkg/cli.RunWorkflowsOnGitHub({0x122afb8, 0x18838c0}, {0xc00004ac70, 0x1, 0x1}, 0x0, 0x0, {0x0, 0x0}, {0x0, ...}, ...) github.com/githubnext/gh-aw/pkg/cli/run_workflow_execution.go:603 +0x634 main.init.func8(0x1852420, {0xc00004ac70, 0x1, 0x1}) github.com/githubnext/gh-aw/cmd/gh-aw/main.go:356 +0x4d9 github.com/spf13/cobra.(*Command).execute(0x1852420, {0xc00004ac40, 0x1, 0x1}) github.com/spf13/cobra@v1.10.2/command.go:1015 +0xb02 github.com/spf13/cobra.(*Command).ExecuteC(0x18512e0) github.com/spf13/cobra@v1.10.2/command.go:1148 +0x465 github.com/spf13/cobra.(*Command).Execute(...) github.com/spf13/cobra@v1.10.2/command.go:1071 main.main() github.com/githubnext/gh-aw/cmd/gh-aw/main.go:630 +0xbd</issue_description>

Comments on the Issue (you are @copilot in this section)


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 2 commits January 26, 2026 15:38
Added mutex synchronization to prevent race condition between Start() goroutine
and Stop()/StopWithMessage() calls. The deadlock occurred when program.Quit()
was called before program.Run() had initialized its channels.

- Added running flag and mutex to track spinner state
- Made Stop/StopWithMessage safe to call before Start
- Made UpdateMessage safe when spinner not running
- StopWithMessage still prints message even if spinner wasn't running
- Added test for rapid Start/Stop to catch race conditions

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix deadlock issue in daily team status command Fix spinner deadlock when Stop called before Start completes Jan 26, 2026
Copilot AI requested a review from pelikhan January 26, 2026 15:44
@pelikhan pelikhan marked this pull request as ready for review January 26, 2026 15:50
@pelikhan pelikhan merged commit e9d6e97 into main Jan 26, 2026
117 of 118 checks passed
@pelikhan pelikhan deleted the copilot/fix-deadlock-in-daily-status branch January 26, 2026 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fatal error: all goroutines are asleep - deadlock! goroutine 1

2 participants