Harden Android DevFlow integration tests#287
Conversation
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Expert Code Review: 3 findings posted inline (2 moderate, 1 minor) plus 1 overflow finding. See the lean summary comment for full details.
Generated by Expert Code Review (auto) for issue #287 · ● 20.1M
- AgentClient: narrow IsTransientTransportException so HttpClient timeout TaskCanceledException no longer triggers retries; only retry HttpRequestException with SocketException inner, IOException, or non-timeout TaskCanceledException. - AndroidEmulatorFixture: parse ps -A PID via positional column 1 instead of the first int-parseable column (custom ROMs may use numeric UIDs in USER). - AndroidEmulatorFixture: short-circuit AVD provisioning when DEVFLOW_TEST_ANDROID_SERIAL points at an online physical device, so the physical-device path no longer requires cmdline-tools to be installed. - WebViewDriverTests: add a 5s CancellationTokenSource and finally-block accept guard to GetStatus_WithTransientRetry_RetriesConnectionRefused so a missed accept can no longer hang the test, and set ExclusiveAddressUse=false on both listeners to reduce port-rebind flakiness. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Thanks for the cross-cutting summary. Addressed all three inline findings (PID parsing, transient retry narrowing, test hang guard) plus the overflow finding: |
There was a problem hiding this comment.
Pull request overview
This PR hardens DevFlow’s Android integration test infrastructure against transient transport instability (notably ADB port-forward drops) to reduce CI flakiness that manifests as cascaded “connection refused” failures while the app is still running.
Changes:
- Added opt-in transient transport retry support to
AgentClientand enabled it for Android integration tests. - Improved Android integration fixture resiliency by re-checking/recreating ADB forwards, debouncing process-loss detection, and adding richer diagnostics capture.
- Added a unit test covering the new retry behavior in
AgentClient.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/DevFlow/Microsoft.Maui.DevFlow.Tests/WebViewDriverTests.cs | Adds a unit test intended to validate transient retry behavior for connection-refused scenarios. |
| src/DevFlow/Microsoft.Maui.DevFlow.Driver/AgentClient.cs | Introduces configurable transient transport retries and applies them across many HTTP operations. |
| src/DevFlow/Microsoft.Maui.DevFlow.Agent.IntegrationTests/Fixtures/IntegrationTestBase.cs | Wraps raw HTTP helpers with Android-only transport retry logic for integration tests. |
| src/DevFlow/Microsoft.Maui.DevFlow.Agent.IntegrationTests/Fixtures/AppFixtureBase.cs | Enables the new AgentClient retry knobs for Android fixtures. |
| src/DevFlow/Microsoft.Maui.DevFlow.Agent.IntegrationTests/Fixtures/AndroidEmulatorFixture.cs | Adds forward re-establishment, debounced process probes, ps -A fallback, and extra diagnostics; supports targeting a physical device via env var. |
- AgentClient: split SendWithTransientRetriesAsync into GET vs explicit-method overloads and add a new RetryMutatingRequests property (default true) gating whether transient retries apply to POST/PUT/DELETE. XML docs spell out the duplicate-side-effect risk on TransientFailureRetryCount and the new RetryMutatingRequests so production callers can opt out cleanly without changing existing integration-test behavior. - IntegrationTestBase: narrow IsTransientTransportException to mirror the AgentClient predicate (HttpRequestException with SocketException inner, IOException, or non-timeout TaskCanceledException) so HttpClient timeouts are no longer amplified into ~8x60s of retries. - WebViewDriverTests: bump the server startup delay to 500ms and wrap the client call in a Stopwatch with an elapsed >= retry-delay assertion so the retry test fails loudly if the retry path is ever skipped. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Follow-up: round 2 of review feedback also addressed two findings this workflow had discarded - the non-idempotent POST/PUT/DELETE retry concern (added a |
Analysis of Finding: Missing serial validation in
|
There was a problem hiding this comment.
Expert Code Review: 3 findings posted inline. See the summary comment for full methodology and discarded findings.
Generated by Expert Code Review (auto) for issue #287 · ● 27.5M
Add opt-in transient transport retries for AgentClient and enable them for Android integration tests to absorb short-lived ADB forward interruptions. Debounce Android app process monitoring, restore missing adb forwards, and capture richer diagnostics when the app appears to disappear. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- AgentClient: narrow IsTransientTransportException so HttpClient timeout TaskCanceledException no longer triggers retries; only retry HttpRequestException with SocketException inner, IOException, or non-timeout TaskCanceledException. - AndroidEmulatorFixture: parse ps -A PID via positional column 1 instead of the first int-parseable column (custom ROMs may use numeric UIDs in USER). - AndroidEmulatorFixture: short-circuit AVD provisioning when DEVFLOW_TEST_ANDROID_SERIAL points at an online physical device, so the physical-device path no longer requires cmdline-tools to be installed. - WebViewDriverTests: add a 5s CancellationTokenSource and finally-block accept guard to GetStatus_WithTransientRetry_RetriesConnectionRefused so a missed accept can no longer hang the test, and set ExclusiveAddressUse=false on both listeners to reduce port-rebind flakiness. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Carry over the more defensive adb forwarding patterns from PR #273: - Scope `adb forward --list` to the active device with `-s <serial>` so parsing tolerates both the 2-column `<local> <remote>` and 3-column `<serial> <local> <remote>` output shapes regardless of how many devices are attached. - Pre-emptively run `adb forward --remove tcp:<agent>` before re-adding the mapping so a stale entry left by a prior adb-server crash cannot block recovery. - After re-establishing the forward, re-list and verify the mapping is visible. `adb forward` can return 0 while the daemon is in an inconsistent state, and silently passing through amplified into a ~4 minute opaque retry storm in the agent client. - Capture both `adb -s <serial> forward --list` and `reverse --list` in the crash diagnostics bundle so port-forwarding state is visible when triaging app-loss artifacts. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- AgentClient: split SendWithTransientRetriesAsync into GET vs explicit-method overloads and add a new RetryMutatingRequests property (default true) gating whether transient retries apply to POST/PUT/DELETE. XML docs spell out the duplicate-side-effect risk on TransientFailureRetryCount and the new RetryMutatingRequests so production callers can opt out cleanly without changing existing integration-test behavior. - IntegrationTestBase: narrow IsTransientTransportException to mirror the AgentClient predicate (HttpRequestException with SocketException inner, IOException, or non-timeout TaskCanceledException) so HttpClient timeouts are no longer amplified into ~8x60s of retries. - WebViewDriverTests: bump the server startup delay to 500ms and wrap the client call in a Stopwatch with an elapsed >= retry-delay assertion so the retry test fails loudly if the retry path is ever skipped. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
7a54bec to
85bbcce
Compare
- AndroidEmulatorFixture: in the 3-column `adb forward --list` fallback, skip rows whose serial does not match the active device. Defensive guard against unexpected multi-device output from older adb releases. - AgentClient / IntegrationTestBase: tighten transient-exception predicates so a bare `TaskCanceledException` with no inner exception is no longer classified as transient. Caller-initiated CancellationToken cancellations must not be retried; only TCEs wrapping a real (non-timeout) transport failure should now flow through the retry loop. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
https://github.com/dotnet/maui-labs/actions/runs/26180361507 Integration tests ran for android on this one too (manually) |
The Android integration CI failure showed the app process still resumed while localhost agent calls started returning connection refused, which points at transient ADB/forward instability rather than a deterministic app crash. This hardens the test transport so hosted-runner blips do not cascade into dozens of false failures.
Changes
AgentClientand enable them only for Android integration fixture calls.adb forwardentries while Android tests are running.pidoftops -Abefore treating the app as gone.DEVFLOW_TEST_ANDROID_SERIALtarget connected physical devices for local debugging.Validation
dotnet test src/DevFlow/Microsoft.Maui.DevFlow.Tests/Microsoft.Maui.DevFlow.Tests.csproj -c Debug --logger "trx;LogFileName=devflow-unit-after-fix.trx"DEVFLOW_TEST_PLATFORM=android DEVFLOW_TEST_ANDROID_SERIAL=38020DLJG004ZG dotnet test src/DevFlow/Microsoft.Maui.DevFlow.Agent.IntegrationTests/Microsoft.Maui.DevFlow.Agent.IntegrationTests.csproj -c Debug --logger "trx;LogFileName=android-full-after-fix.trx" --results-directory artifacts/TestResults/devflow-integration/android-full-after-fix