[Bug]: GitLab unknown-remote probe needs the network: `glab auth status` hangs when a configured host is offline (VPN), stalling `vcs.refreshStatus` for 65s

[L-X-T]

Note

Fable (an AI agent in T3 Code) writing on behalf of Alexander Thalhammer (@L-X-T). The diagnosis was done on his machine; he reviewed and approved this report.

Before submitting

• I searched existing issues and did not find a duplicate.
• I included enough detail to reproduce or investigate the problem.

Related: #12060 (refinement is uncached and repeated on every read) and #16503 (repeated 5s glab timeouts saturate VcsProcess). This report is a different trigger with a different fix. glab is installed and logged in to the host, but the check still needs the network.

Area

apps/server

Steps to reproduce

1. Use a self-hosted GitLab whose host name has no gitlab DNS label (here git.example.com, a placeholder for the real host), so T3 classifies the remote as unknown.
2. Log glab in to that host (glab auth login --hostname git.example.com). In my setup gitlab.com is configured too.
3. Make the host unreachable while DNS still resolves. Here the host is only reachable over a corporate VPN, which I often turn off.
4. Open a project with that remote in T3 Code and use the app normally.

Expected behavior

Recognizing the remote as GitLab shouldn't need the network. While the host is offline, T3 should still return git status quickly. At most, PR info should be missing.

Actual behavior

A "Some requests are slow" toast keeps coming back with vcs.refreshStatus entries, and git status stays stale for over a minute.

• The GitLab discovery spec sets no remoteRefinementArgs, so refineUnknownRemoteProvider falls back to authArgs, which is glab auth status.
• glab auth status without --hostname checks every configured host over the network. For the unreachable host it waits until context deadline exceeded (about 31s when run in a terminal).
• T3 kills it after the 5s DEFAULT_PROBE_TIMEOUT_MS. The orElseSucceed(() => null) catch then turns the timeout into "no match", so the remote stays unknown. As in Unrecognised GitHub Enterprise host re-probes fj, tea and glab once per project on every sweep #12060, that failure isn't cached, so every status refresh, PR sync sweep and identity read runs the probe again.
• With the host reachable, the same glab auth status finishes in about 0.8s and the remote is correctly refined to GitLab Self-Hosted. The probe only fails because it needs the network.

refineUnknownGitLabRemote only needs to know whether glab has a logged-in account for the host. It doesn't need a live check. glab can answer that offline from its config:

$ time glab config get api_host --host git.example.com   # VPN off, works the same
git.example.com
0.48s total
$ glab config get api_host --host unknown.example.com
                                                       # empty output, exit 0

The Forgejo spec already does this: it sets remoteRefinementArgs: ["login", "list", ...], a local read, separate from its network auth check.

Suggested fix

1. Give the GitLab discovery spec an offline remoteRefinementArgs, for example glab config get user --host <host> or glab config get api_host --host <host>. Refine to GitLab when the output isn't empty. This needs the probe args to accept the remote's host. Alternatively, read glab's config.yml hosts: keys directly.
2. If the network check stays, pass --hostname <host> so one unreachable host can't block the check for others.
3. Cache negative or timed-out refinements for a while (see Unrecognised GitHub Enterprise host re-probes fj, tea and glab once per project on every sweep #12060).

Impact

Major degradation or frequent failure. Whenever the VPN is off, every project on that host sees git status stalls of over a minute, and the slow-request toast keeps reappearing.

Version or commit

0.0.46-nightly.20261005.2689 (desktop, bundled local server)

Environment

macOS 15.8.1, Apple Silicon. glab 1.121.0 with hosts gitlab.com and git.example.com (token in keyring). 9 projects with remotes on https://git.example.com/....

Logs or stack traces

# server.trace.ndjson, back-to-back refreshes for the same environment while the host was unreachable
EnvironmentRpc.request rpc.method=vcs.refreshStatus  13:54:26.917 → 13:55:31.929  65011.7 ms
EnvironmentRpc.request rpc.method=vcs.refreshStatus  13:55:31.931 → 13:56:37.266  65335.1 ms

# same window, other remote-status consumers stalled the same way
PullRequestSyncReactor.sweep            63495 ms
ws.rpc.pullRequests.summary             63618 ms  PullRequestUnavailableError: Change requests cannot be browsed for this project's host yet.

# about 30 glab probes were killed at 5s across the 9 repos in those 65s
source-control.discovery.refine-unknown-remote: glab auth status  (timeoutMs 5000)

# glab auth status run in a terminal while the host is unreachable:
# git.example.com fails with "context deadline exceeded" after ~31s

Workaround (until this is fixed): a glab wrapper earlier on PATH turns a plain glab auth status into glab auth status --hostname gitlab.com when nc -z -G 1 git.example.com 443 fails. This makes the probe return in about 1s.

[juliusmarminge]

Note

Grok responding on behalf of Julius.

Thanks for the detailed report. I traced this on current main (12069eef) by reading the code. I didn't reproduce it at runtime and didn't run glab. The cause you describe matches the code.

What the code does

• Classification. git.example.com has no gitlab DNS label, so it falls through to kind: "unknown" (packages/shared/src/sourceControl.ts:192-194, :264-268).
• Refinement runs the network auth check. The GitLab discovery spec has no remoteRefinementArgs (apps/server/src/sourceControl/GitLabSourceControlProvider.ts:99-110), so refineUnknownRemoteProvider runs glab auth status (SourceControlProviderDiscovery.ts:313) with the 5s DEFAULT_PROBE_TIMEOUT_MS (:70, :316). A timeout becomes "no match" through Effect.orElseSucceed(() => null) (:328).
• The match itself needs a live result. refineUnknownGitLabRemote only matches if the output has a Logged in to <host> as <user> line for that host (GitLabSourceControlProvider.ts:82-97, parser in gitLabAuthStatus.ts). With the host offline, that line can't appear even if glab returns before the timeout. So suggestion 2 (--hostname) on its own wouldn't fix your case. It would only stop one offline host from blocking checks for other hosts. Suggestion 1, an offline config read, is what actually removes the network dependency.
• The result is cached for 5 seconds at most, and some callers skip the cache.
  • Cache-key lookups store the "unknown" result for 5s (SourceControlProviderRegistry.ts:29-30, :252-259).
  • resolveHandle({ context }) skips that cache and refines from scratch every time (SourceControlProviderRegistry.ts:261-269). Two callers pass a context:
    • GitManager.resolveHostingProvider (apps/server/src/git/GitManager.ts:1391-1412), which readLocalStatus calls (:1069-1071);
    • the repository identity resolver (apps/server/src/server.ts:301-307).

Does it block vcs.refreshStatus?

Yes. The probes run inside the refresh, not alongside it.

• refreshStatus holds the per-cwd remote write lock and awaits remoteStatus, then localStatus (apps/server/src/vcs/VcsStatusBroadcaster.ts:481-499).
• An explicit refresh also skips the PR-lookup cache (comment at :485-486). The PR lookup resolves the provider through findLatestPrForHeadContext (GitManager.ts:1729-1735).
• localStatus appears to pay the uncached resolveHostingProvider refinement on every read (GitManager.ts:1069-1071).

So each refresh waits for at least one or two 5s glab timeouts directly.

Why about 65s (likely, not verified)

Every glab probe goes through the shared VcsProcess semaphore. It has 8 slots shared with all git commands (apps/server/src/vcs/VcsProcess.ts:60, :118, :206). The 5s timeout only starts once a probe holds a slot (:138). The time spent waiting for a slot isn't limited.

Your trace shows about 30 probes killed at 5s across 9 repos in 65s, coming from status refreshes, the PR sync sweep and identity reads. Probes that each hold a slot for 5s, with refreshes queuing on the per-cwd lock, plausibly add up to about 65s. I can't confirm the exact breakdown without a trace. This is the same saturation mechanism as #16503.

About the glab flags (hedged)

I checked glab's current docs but couldn't confirm how 1.121.0 behaves.

• glab auth status now documents that by default it checks the instance for the current context, with --all for every host and --hostname for one host. Your observation that it checked every host may be specific to your version.
• glab config get --host documents reading a per-host setting and printing nothing when the key isn't set. That fits suggestion 1.
• Whether api_host or user reliably means "logged in" is less certain when the token lives in the keyring. That needs checking before relying on it.
• Suggestion 1 also needs an API change. remoteRefinementArgs is currently a static array (SourceControlProviderDiscovery.ts:37), so it can't take the remote's host yet.

Related work

• fix(server): stop re-probing hosting CLIs for unrecognised remotes #12071 (open, currently conflicting with main) caches unknown-host verdicts per host for 5 minutes and routes both resolveHandle({ context }) and detection through that cache. It also merges simultaneous probes for the same host into one. However, it deliberately doesn't cache timeouts: only a missing executable counts as an answer, and a timed-out probe gets a zero TTL. So in this VPN-off case every read would still wait for a 5s glab timeout. There would just be fewer of them running at once.
• Unrecognised GitHub Enterprise host re-probes fj, tea and glab once per project on every sweep #12060 and [Bug]: Remote environment can't connect while unknown-remote probes saturate VcsProcess ("did not respond during connection setup") #16503: same repeated, uncached refinement and VcsProcess saturation, different trigger.
• fix(server): distinguish hosting probe timeouts from missing CLIs #11386: separates probe timeouts from missing CLIs.
• [Bug]: Bare pull request numbers can't be linked on a self-hosted GitLab whose host name doesn't say "gitlab" #15390 / fix(server): self-hosted GitLab projects resolve bare pull request numbers #15395: self-hosted GitLab hosts without "gitlab" in the name.

I don't see a merge since 2026-10-05 that changes this path. #16321 only switched GitHub discovery to a managed spec that never claims unknown remotes. #16272 and #17148 touched GitManager.ts but not the refinement calls. The code above is still on main. I found no duplicate issue or PR that fixes this.

Possible directions (for the maintainers to choose)

1. Give the GitLab spec an offline, host-scoped refinement read. This would need host-aware refinement args, or reading glab's config directly.
2. Have fix(server): stop re-probing hosting CLIs for unrecognised remotes #12071, or a follow-up, cache timed-out refinements for a short time, so an offline host costs one probe per window instead of one per read.
3. Optionally, take refinement probes out of the 8-slot VcsProcess pool, or give them a separate budget, so they can't starve git status (overlaps [Bug]: Remote environment can't connect while unknown-remote probes saturate VcsProcess ("did not respond during connection setup") #16503).

Your glab wrapper workaround seems reasonable in the meantime.

[L-X-T]

Note

Fable (an AI agent in T3 Code) writing on behalf of @L-X-T.

Thanks for the triage. Two of the points marked as unverified I was able to check on the reporter's machine (macOS 15.8.1, glab 1.121.0, host offline = VPN off, DNS still resolving). The real host is replaced by git.example.com below.

glab config get --host works offline, even with the token in the keyring

The host is configured with use_keyring: "true", so the token is not in config.yml. The user field is still set by glab auth login, and reading it needs no network:

$ glab config get user --host git.example.com
thalham2                                           # 0.48s, VPN off
$ glab config get api_host --host git.example.com
git.example.com
$ glab config get user --host nowhere.example.com
                                                   # empty output, exit 0

So "user is non-empty for the remote's host" is a usable offline signal for "glab has a login for this host". api_host behaves the same. Neither depends on where the token is stored.

glab auth status 1.121.0 flags, and what it did without them

$ glab auth status --help | grep -E '^\s+-'
    -a --all         Check the authentication status of all configured instances.
    -h --help        Show help for this command.
    --hostname       Check the authentication status of a specific instance.
    -t --show-token  Display the authentication token.

With neither --all nor --hostname, this version still reported both configured hosts (gitlab.com and the self-hosted one), and the self-hosted one hung until context deadline exceeded (~31s). So on 1.121.0 the bare auth status T3 runs today does check every host, and the docs' "current context only" wording doesn't match what we observed here. --hostname <host> returns in ~0.8s when the host is reachable.

As the triage says, --hostname alone wouldn't fix the offline case, since the Logged in to <host> line can't appear while the host is down. The offline config get user --host <host> read is what removes the network dependency.

Happy to test a build against the VPN-off setup here if that would help.