Skip to content

An Azure DevOps build can stay "Running" after its run completes, because the completed FinishTime is older than the local UtcNow stamped on the last in-progress poll #310

Closed

Description

@matt-edmondson

What's wrong

AzureDevOps.UpdateRunFromBuild (BuildMonitor/Providers/AzureDevOps.cs:~365) mixes two clocks when it sets run.LastUpdated:

// For LastUpdated, use current time if not finished yet (in-progress builds)
run.LastUpdated = build.FinishTime ?? DateTimeOffset.UtcNow;
  • While the build is running, LastUpdated is the local time of the poll.
  • Once it completes, LastUpdated is the server's FinishTime.

Build.UpdateFromRun (BuildMonitor/Build.cs:~76) then accepts a new status for the same run only when the timestamp has moved forward:

else if (run.Started == LastStarted && run.LastUpdated > LastUpdated)

If the server's FinishTime is earlier than the local time stamped on the last in-progress poll, Build never takes the completed status. FinishTime never changes after that, so the build stays wrong until a newer run starts. This happens when:

  • the local clock is ahead of the Azure DevOps server's clock, even by a second, or
  • the build finishes while an in-progress response is still in flight, so the poll is processed after FinishTime. BuildSync and RunSync both poll the same run, which makes this window bigger.

Failure scenario

  1. Poll 1 returns the build as InProgress. It is processed at local time T, so Build.LastUpdated = T and LastStatus = Running.
  2. Poll 2 returns Completed/Succeeded with FinishTime = T − 1s.
  3. run.Status becomes Success, but Build.UpdateFromRun ignores it because T − 1s > T is false.

The subagent that reviewed this reproduced it with a scratch MSTest case. The case calls the private UpdateRunFromBuild through reflection with those two build snapshots:

run.Status=Success build.LastStatus=Running build.IsOngoing=True

Effect in the UI: the build row keeps showing a progress bar and ETA and stays on the fast (high-priority) poll interval, even though its latest run has finished. Its row in the runs list correctly says Success.

Suggested fix / acceptance criteria

Either of these works:

  • In Build.UpdateFromRun, for the same run (run.Started == LastStarted), always accept a change from an ongoing status to a finished one, whatever the timestamps say.
  • Stop using the local clock in the ADO provider. Keep LastUpdated on server times only, or track the latest run's id on Build and always take that run's status.

Add a test that feeds an in-progress snapshot and then a completed snapshot whose FinishTime is earlier than the in-progress poll's local timestamp, and checks that Build.LastStatus ends up Success and IsOngoing is false.

Activity

  1. matt-edmondson commented on Sep 28, 2026

    @matt-edmondson
    ContributorAuthor

    Triage


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

bugSomething isn't workingreadyFully specified; implement as written

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions