Repository navigation
[tooling] Replace ProcessUtils with native .NET 11 APIs - #12988
Draft
simonrozsival wants to merge 22 commits into
Draft
simonrozsival wants to merge 22 commits into
simonrozsival wants to merge 22 commits into
Conversation
`Xamarin.Android.Tools.ProcessUtils` already drains both output streams concurrently, but inherited pipes, cancellation, and consumer faults could keep its task pending or leave the root process running. Separate root exit from EOF and bound post-exit draining and shutdown. Preserve writer ownership, text capture, callbacks, and public APIs. Propagate parser failures from ExecuteToolAsync and add focused, target-compatible synthetic-process regressions. Related to #12981 (process audit group 4). No framework retargets or caller migrations; keep persistent descendants out of root cleanup. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Make the Android SDK fixture runtime floor explicit and provision a .NET 11 SDK immediately before its CI test invocation. Keep the other Android tooling builds and base-task tests on their existing host. This narrow prerequisite is shared by process-audit groups 2 and 4. It changes no product target, fixture entry point or packaging graph. Related to #12981. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Complete the SDK test-owner host prerequisite by moving only the SDK library and ls-jdks build after native-capable SDK provisioning. The benchmark and base-task steps retain their existing host. This template-only prerequisite is shared with process-audit group 4; that owner remains responsible for its product/reference floor. Related to #12981. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Call the .NET 11 raw-text reader at the apkdiff owner instead of the SDK ProcessUtils facade. Keep its raw diagnostics, nonzero status, log attachment and partial output after the actual execution timeout. Bound EOF separately and terminate only the owned process tree. Require net11.0 for the host harness and its direct device-test/tool closure. Update their real artifact paths and invoke the SDK already provisioned by their CI setup/build jobs; unrelated test owners retain their existing host. Do not move global or product framework aliases. This narrow caller/floor prerequisite is shared with audit group 4. Other host implementation and regression changes remain separate. Related to #12981. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The native raw-text reader takes synchronous ownership of both streams. Process.Dispose does not close readers in that mode, so explicitly own and dispose them after awaited exit cleanup on every terminal path. Keep this correction separate for the shared facade-removal prerequisite. Related to #12981. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Delete the shared SDK ProcessUtils API and its compatibility/helper test infrastructure. Execute, capture, and stream at actual command owners with the native .NET 11 APIs, preserving ADB server/emulator ownership, interactive input, diagnostics, and streamed instrumentation results. Wire the necessary SDK/product/installer/tool/selected Java.Interop runtime and output-path closure, without changing global aliases or unrelated BaseTasks/Installer.Common/Java.Interop target frameworks. Existing canonical test-owner prerequisites are integrated separately. The .NET 11 annotated-BCL nullable adaptations exposed by the product compiler are a separate, necessary follow-up commit for review. Related to #12981. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The native SDK dependency requires this task owner to target .NET 11. The annotated BCL exposes 171 nullable diagnostics in 80 source files. Correct the exact input, XML, path, dictionary and interface contracts without weakening nullable checks or adding null-forgiving operators. Use the native AAPT2 input-encoding property directly. Remove its obsolete reflection, console-encoding fallback, and coordination lock. Keep the daemon job protocol and UTF-8-without-BOM input unchanged. The clean product and host-test rebuilds pass strict warning checks. Related to #12981. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Guard only the two Windows CSP fallback blocks so other platforms keep the original cryptographic failure. The converter has one source-link compile owner, which now targets .NET 11. Four host-side manifest converters read Android enum constants as data. Scope justified CA1416 exceptions to those methods only; do not mark the desktop host Android-only or disable platform checking globally. The real strict product rebuild passes without warnings or errors. A macOS compiled-owner probe verifies all four converters, an existing signing key, and both invalid-key failures without a Windows fallback. Windows fallback execution remains unverified locally. Related to #12981. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The existing 500 ms boot deadline can kill the new shell process before it executes the fake emulator on a busy host. Let the fake device become online only after its argument file exists, using the established SDK fixture pattern and a bounded deadline. Keep dedicated timeout and cancellation tests unchanged. The selected SDK suite passes 267 tests with three Windows-only skips; a separate emulator repeat passes all 27 cases. Related to #12981. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
simonrozsival
changed the title
The pinned SDK12 preview maps the net11 moniker to its own runtime and targeting pack. Select the real .NET11 runtime and references at this test owner so its runtimeconfig and compilation match the declared floor. Reproduce the emitted framework12 requirement, then build with exact 12.0.100-alpha.1.26477.101 and run the established SDK fixtures on a host containing only 11.0.0-rtm.26479.103. Keep this shared prerequisite separate for process-audit group 4; no product or global SDK pin changes. Related to #12981. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The exact CI SDK 12.0.100-alpha.1.26477.101 maps nominal net11.0 projects to its own runtime and targeting pack. Run consequently asks for .NET 12 and fails with exit 150 on the declared MAUI .NET 11 host. Select runtime and framework references 11.0.0-rtm.26479.103 at each native-tooling owner, with an owner-scoped restore feed. Include only the required existing trimmable-map compile dependency, not its source workstream. Preserve global pins, unrelated TFMs and feature settings. Clean strict SDK12 builds pass. The real shipped Run and ls-jdks tools execute on the isolated .NET 11-only host. Native SDK, NativeAOT and assembly-store tests pass with an explicitly selected .NET 11 testhost; metadata and resolved-pack checks confirm actual version 11 references. Related to #12981. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The exact CI SDK12 preview maps net11.0 to its own runtime and targeting pack. Select the real .NET11 runtime and reference pack at the native harness owners so their declared host floor and output paths are true. Use the scoped dotnet11 feed, not a global SDK or product graph change. Reproduce the .NET12 selection, inspect all five owners' resolved Core references and runtimeconfigs, then build with the exact pinned SDK. Run 19 retained native policies and 31 SDK fixture tests on an actual .NET11-only host. Full prepared-SDK/device execution remains separate. Related to #12981. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Honor the six upstream source retirements rather than resurrecting nullable-only edits to the removed pipeline. Preserve the explicit native .NET 11 runtime/reference floor and shared SDK dependency while keeping the new isolated-task helpers and packaging changes from main. The merge introduces one clang fixture consumer of deleted ProcessUtils. Call native capture directly there, with structured arguments and the same stdout/stderr failure diagnostics. Add no replacement runner. Clean strict product and modern-task builds pass; 74 retained native and synthetic target cases pass on the SDK-capable host. Canonical fixture-host and linked compression package prerequisites are separate. Related to #12981. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The shared MSBuild imports no longer carry K4os after #12983. Both host/device test assemblies link AssemblyCompression.cs, whose .NET11 branch uses LZ4. Declare the existing tooling version directly at those owners instead of restoring an unused shared product dependency. Fresh restore and focused Csc checks confirm both NET11_0_OR_GREATER branches compile with the explicit1.3.6 reference. Existing LZ4/Zstd regressions pass; the current third-party notice already covers K4os. Related to #12981. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
UseDotNet11.0.x selected RC1, which cannot launch the SDK fixtures requiring 11.0.0-rtm.26479.103. Provision the repository-pinned SDK with the existing installer, then build and run only the native SDK owners through the existing explicit-host template. Select that provisioned executable for VSTest too and preserve TRX publishing on failure. Benchmark and BaseTasks steps retain their existing host. The declared .NET11 runtime/reference floor is unchanged; no older-runtime fallback or cloud-CI iteration is used. Execute the actual provisioning/build/ fixture flow locally before publishing this correction. Related to #12981. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Document that fixture CI provisions the existing pinned SDK and selects its executable for VSTest. A wildcard preview installation can be older than the explicit .NET 11 runtime floor, while unrelated tooling owners retain their own host selection. Related to #12981. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Build1622808 uses an SDK10 parent that either retains dotnet after Windows provisioning or passes its SDK selection to new SDK12 children. The native SDK/JI owners then fail NETSDK1045 on all three platforms. Integrate precisely the coordinated two-script prerequisite from group1. Resolve the installed executable after provisioning, bind child SDK context explicitly, and execute the inventory on that provisioned host. Do not include unrelated generator floor changes or global pin edits. Reproduce the inheritedSDK10 failure locally; the real Prepare body with an SDK10 parent and late-provisionedSDK12 then passes, including actual bootstrap/inventory builds and verified native child contexts. Related to #12981. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
WildcardUseDotNet11 selectedRC1, below the declared11RTM floor, so both Bytecode and Java.Interop testhosts abort before reporting results. Provision the existing repository-pinned SDK, use the common explicit host invoker for preparation/builds and native tests, and select its platform executable for VSTest. Keep legacy test hosts/TRX/retries intact. Actual JI preparation and solution build pass on the provisionedSDK12. Strict affected-owner builds pass; Bytecode98tests and Java.Interop736 pass, with seven expected skips and verified explicit worker paths. Runtime/reference requirements remain true11RTM; no policy rollback, new process runner or cloud-CI iteration is introduced. Related to #12981. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The all-at-once raw capture API discards its tuple when an inherited pipe outlives apkdiff. Capture both readers concurrently into owned raw-text buffers, append the EOF diagnostic and retain the completed process status independently of the output deadline. Keep this owner correction separate for the shared caller prerequisite. The established regressions reproduce exit0 and7 being replaced by -1, then verify concurrent128KiB raw streams, CRLF, blank lines and tails remain intact after the two-second EOF bound. Related to #12981. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The canonical EOF correction uses interpolated StringBuilder.Append handlers. The linked device-test owner treats CA1305 as an error, unlike the host suite. Specify the existing current-culture contract on only the two diagnostic appends instead of disabling the analyzer. Both actual owner projects pass strict builds. Five focused cases in the established host suite verify raw CRLF/blank lines/tails, concurrent 128KiB output with inherited handles and real exit0/7, log shape, and the actual30second execution deadline retaining partial diagnostics and terminating its owned child. The declared true11 floor is unchanged. Related to #12981. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Merge published main7ef934d47 normally and preserve its retired native typemap support. Resolve the sole task-registration conflict by keeping the required modern host and dropping the now-deleted empty-stub task. Do not restore Full-MSBuild compatibility flags or obsolete source. The real merged product and linked host/device-test owners pass strict builds. Required packaging manifest corrections are a separate commit. Related to #12981. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
All three packaging lanes in build1622829 fail NU5019 on the absent System.Collections.Immutable.dll. The native tooling floor no longer copies its former support package, and System.Reflection.Metadata is provided by that same host framework rather than a shipping package. Remove exactly those two stale installer entries instead of restoring obsolete runtime packages. Keep System.Buffers, System.CodeDom and System.IO.Hashing because real unchanged owners still produce them. The actual full SDK project reproduces the failure locally before this change, then creates complete Darwin, Windows and Linux SDK and symbol NuGets with zero warnings/errors. Inspect all payloads for native paths, required remaining dependencies, matching flat/net SDK bytes and the actual .NET 11 runner runtimeconfig. Packaged Run executes on11-only. Related to #12981. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Important
Parked until .NET 11 becomes the repository's shared stable TFM. This is the user's confirmed scope decision. These native-API changes assume the central stable build/test hosts will support .NET 11 after that separate transition. The repository currently still uses
net10.0, so native-API compilation and CI failures before the transition are expected and accepted. Keep this PR draft; revisit it only after the central stable-.NET-11 transition. Do not add owner-specific runtime/reference-pack/feed overrides, special CI provisioning/hosts, or older-runtime fallbacks to make it green beforehand. Previously recorded configured-head verification is historical evidence, not a reason to resume pre-transition CI repairs.Related to #12981 (process-execution audit, group4). This does not close the full audit.
Delete
ProcessUtils, not replace it. SDK, installer, debugging, instrumentation and NativeAOT command owners use purpose-built .NET11 APIs directly. Facade API declarations/tests and the dedicated child project are removed. No renamed generic runner or older-runtime adapter remains.Application policies stay at their owners: ADB environment/structured arguments/diagnostics; root exit versus EOF with a30-second post-exit bound; cancellation of only the owned client, not shared servers; interactive AVD/license stdin; serialized instrumentation/LLVM parsing; explicit reader ownership and supplied-writer preservation. Native capture retains CR/LF/tails with the selected encoding. Persistent emulator launch retains its caller-owned process and signal policy.
Runtime/reference/owner closure
Affected SDK/product/debugging/installer/config/run/discovery/assembly-store owners and selected Java.Interop bootstrap/test consumers require compatible .NET11-or-later CLI/MSBuild hosting with truthful assembly paths. Runtime and reference pack select 11.0.0-rtm.26479.103: pinned SDK 12.0.100-alpha.1.26477.101 otherwise maps nominal net11.0 to its own runtime/pack12. A reproduced Run binary required12 and failed on actual11 with exit150 before correction. Real pack paths, System.Runtime/Process version11 identities, runtimeconfigs and actual11-only execution are verified—not just labels.
Global SDK/framework pins/aliases, BaseTasks, Installer.Common and unrelated Java.Interop requirements remain unchanged. Flat tools/ and isolated tools/net/ layout and matching native SDK/trimmable dependency copies are preserved. The separate bounded nullable prerequisite corrected171 original annotated-BCL diagnostics in80 XABT files without weakened checks or null-forgiving operators. AAPT2's obsolete reflection/console-encoding fallback/lock is replaced by StandardInputEncoding without changing its protocol. Separate platform fixes guard only two Windows CSP fallbacks and scope justified CA1416 annotations to four desktop enum-to-manifest converters.
Canonical group2 floor/host/codec prerequisites are integrated as isolated commits rather than their whole branch. No product shared1.1.11 K4os dependency is restored; the two linked-source test owners declare the existing1.3.6 pin and notice. SDK fixture CI uses the existing pinned installer/common invoker and explicit VSTest executable, preserving unrelated benchmark/BaseTasks hosts/TRX. Fake-emulator readiness retains the real SIGINT assertion without changing product behavior or excluding CI tests.
Latest packaging correction and current-main merge
All three failures in build1622829 share the same first real error, not three speculative fixes. That build tested fc2ee12, via merge SHA b39ee9e; it did not test the later APKDiff review head.
make create-installers, log141bin/Release/lib/packs/Microsoft.Android.Sdk.Darwin/37.99.0/tools/System.Collections.Immutable.dllwhile buildingbuild-tools/create-packs/Microsoft.Android.Sdk.proj.make create-nupkgs, log204Microsoft.Android.Sdk.Linux/37.99.0/tools.Test PackDotNet, log252; subsequent failure gateMicrosoft.Android.Sdk.Windows/37.99.0/tools.The later MSB3073/code2/code1 messages are cascades of this SDK-pack failure. All four SDK/JI jobs passed and4480 tests passed/60skipped with zero recorded failed assertions; this is a migration packaging defect, not a test flake.
58dddf4 removes exactly two stale shared installer items: System.Collections.Immutable.dll and System.Reflection.Metadata.dll. The native owner now resolves them from the host reference/framework pack (
CopyLocal=false); no real remaining output producer supplies those old support-package files. Re-adding obsolete packages or using Exists filters would hide the closure defect. System.Buffers.dll remains because unchanged netstandard BaseTasks/Installer.Common owners still produce it; System.CodeDom/System.IO.Hashing and all native paths remain packaged. The directly related SDK documentation explains this distinction.b6cb541 is a separate normal merge of current published main 7ef934d. Its only conflict was trimmable task registration: preserve the modern host requirement and honor upstream's deleted empty-stub/native-typemap support. No retired source/runtime is resurrected, no Full-MSBuild compatibility flags are added, and upstream native/compiler workstreams are not redesigned. Prior merge
1c89261f78already honored the six earlier retirements.Exact latest packaging verification
macOS arm64, actual repository-provisioned SDK12.0.100-alpha.1.26477.101 in this worktree's bin/Debug/dotnet. No source SDK pin, suppression, feature disabling, new harness or cloud-CI iteration.
Identical failure reproduced locally, using the real complete SDK package project rather than a mock item subset:
DOTNET_ROOT="$PWD/bin/Debug/dotnet" ./bin/Debug/dotnet/dotnet build build-tools/create-packs/Microsoft.Android.Sdk.proj -c Debug -p:HostOS=Darwin -v minimalBefore the two-item correction: exit1 / NU5019, exact missing tools/System.Collections.Immutable.dll. Ordinary missing local preparation inputs were generated by their existing owners, not faked or disabled:
After correction, all three real complete SDK and symbol NuGets passed with0warnings/0errors:
No subset filtering or placeholder files were used. The final packages are version
37.99.0-ci.simonrozsival-shared-process-runner-fixes.112, with394 Darwin,313 Windows and395 Linux archive entries, plus their symbol packages. Native Windows/Linux execution and the outer MSI/installer workflow are not claimed merely because their complete SDK NuGets were packed on macOS.Payload assertions passed for every actual final archive: both retired DLLs absent; required flat/net task/SDK/trimmable paths present; genuine remaining System.Buffers/CodeDom/IO.Hashing retained; flat/net SDK bytes identical; no tools// path artifacts; correct package IDs; Run runtimeconfig still net11.0 /11.0.0-rtm.26479.103. A safely extracted final Darwin payload's Run DLL executed successfully on the provided actual11-only host:
Exit0; both retired support DLLs were absent during execution. Temporary extraction was cleaned afterward. Archive proof, baseline/after logs and actual CI task logs/artifact paths are retained in session files (
packaging-final-archive-proof.json,packaging-published-*.log,packaging1622829-*.txt).Strict real merged owner builds passed0warnings/0errors, using existing preparation dependencies where BuildProjectReferences=false is specified:
Passed47/47 retained product utility/manifest cases on actual11 after the latest main retirement; one more upstream legacy-only case is now removed, not skipped by this PR:
Retained canonical APKDiff correction
Canonical isolated group2 commit5f3feb6b is integrated as cb045ab, not the mixed host-review commit. Inherited-pipe EOF cancellation no longer discards raw diagnostics or replaces completed exit0/7 with-1. Concurrent cancellable native reader chunks preserve partial buffers; EOF diagnostics append separately from real status. Actual execution timeout remains-1 with owned-tree cleanup, existing trimming/log/attachment/reader ownership unchanged. 22a26ca explicitly preserves current-culture formatting for the two diagnostic appends, fixing strict linked-device CA1305 without suppression.
Both actual owners compiled strictly and 5/5 focused real-owner cases passed on actual11: raw CRLF/blank lines/tails/log0&7; simultaneous128KiB stdout/stderr retaining0&7 at the2-second inherited EOF bound (~2.35s); actual30-second deadline retaining partials/terminating child (~30.05s). Scratch-only cases were linked into the existing NUnit suite, with no committed harness or production knob; normal outputs were rebuilt afterward:
Earlier setup repair and evidence
Build1622808 failed in setup, with no recorded test assertion failures: product3OS children inheritedSDK10 despite newSDK12, and JI native workers receivedRC1 below11RTM. 418f69e integrates the exact coordinated group1 two-script prerequisite (no generator/whole-branch changes): resolve executable after installation, bind child SDK/MSBuild/host context, inventory on the provisioned host. fc2ee12 uses existing pinned installer/common invoker for JI setup/builds and selects its worker for only two net11 tests, preserving net10/TRX/retries.
Local SDK12+inheritedSDK10 reproduces NETSDK1045/exit1 before. The actual Prepare body from an SDK10 parent with a late-provisioned prefix passes, including real bootstrap/inventory0warnings/0errors; unrelated Android/JDK downloads are intercepted only in that bounded scratch proof. Temporary prefix/workers were cleaned:
Actual JI preparation/solution build passed; full solution had10 retryable shared-output copy warnings/0errors, strict affected native owners0warnings/0errors. Full Bytecode98passed/2skips and JNI736passed/5skips pass on provisioned12 and actual11-only workers; runtimeconfigs retain11RTM. CI1622829 subsequently confirms all four SDK/JI jobs pass on Windows/macOS; the packaging layer above was the remaining red gate. Prior retained checks also passed74 native/synthetic MSBuild cases and11 LZ4/Zstd cases. Local SDK full-suite had one pre-existing PATH/preferred-JDK environment failure; CI stays unfiltered and SIGINT passes.
Remaining gaps: fresh CI for this new head, native Windows/Linux execution of the latest package/merge changes, outer full installer/MSI/device/workload execution, Windows CSP fallback and other platform samples. The three diagnosed SDK-pack failures are reproduced/fixed locally with complete packs, not represented as already-green cloud CI. Current-review-head checks without an Azure URL are not conflated with older build1622829. No PR completion/merge, rebase/force-push/amend, extra session/PR, global-floor weakening or cloud-CI rerun loop.
Pull Request
title and
description
should follow the
commit-messages.mdworkflow documentation, and in particular should include: