Skip to content

[tests] Update max simulator OS version.#24867

Open
rolfbjarne wants to merge 4 commits intomainfrom
dev/rolf/update-max-simulator-version-logic
Open

[tests] Update max simulator OS version.#24867
rolfbjarne wants to merge 4 commits intomainfrom
dev/rolf/update-max-simulator-version-logic

Conversation

@rolfbjarne
Copy link
Member

This fixes the following problem:

  • We default the max simulator version to the SDK version of an OS (currently 26.2)
  • However, iOS/tvOS 26.3 exists, and that's the simulator 'xcodebuild -downloadPlatform' installs with Xcode 26.3.
  • The max simulator version is what we use in xharness to decide which simulator to run tests in.
  • The result is that xharness will try to use/create a simulator for iOS 26.2, but that might not be possible (unless said simulator was already installed from before the current build), because it wasn't provisioned.

This fixes the following problem:

* We default the max simulator version to the SDK version of an OS (currently 26.2)
* However, iOS/tvOS 26.3 exists, and that's the simulator 'xcodebuild -downloadPlatform' installs with Xcode 26.3.
* The max simulator version is what we use in xharness to decide which simulator to run tests in.
* The result is that xharness will try to use/create a simulator for iOS 26.2, but that might not be possible (unless said simulator was already installed from before the current build), because it wasn't provisioned.
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates simulator test OS selection so xharness uses the newest simulator runtime available with the selected Xcode (even when the SDK version hasn’t changed), avoiding attempts to create/run simulators for unprovisioned OS versions.

Changes:

  • Introduce logic in Make.config to set MAX_IOS_SIMULATOR_VERSION / MAX_TVOS_SIMULATOR_VERSION based on XCODE_VERSION (special-casing Xcode 26.3 -> simulator 26.3).
  • Add make-time guardrails ($(error ...)) to force revisiting this mapping when Xcode/SDK version relationships change.

You can also share your feedback on Copilot code review. Take the survey.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@rolfbjarne rolfbjarne enabled auto-merge (squash) March 10, 2026 14:58
@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2
Copy link
Collaborator

✅ [CI Build #e5b4558] Build passed (Build packages) ✅

Pipeline on Agent
Hash: e5b4558abecc86b86bd4dee307942a78bd525c70 [PR build]

@vs-mobiletools-engineering-service2
Copy link
Collaborator

✅ [PR Build #e5b4558] Build passed (Detect API changes) ✅

Pipeline on Agent
Hash: e5b4558abecc86b86bd4dee307942a78bd525c70 [PR build]

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2
Copy link
Collaborator

✅ [CI Build #e5b4558] Build passed (Build macOS tests) ✅

Pipeline on Agent
Hash: e5b4558abecc86b86bd4dee307942a78bd525c70 [PR build]

@vs-mobiletools-engineering-service2
Copy link
Collaborator

✅ API diff for current PR / commit

NET (empty diffs)

✅ API diff vs stable

NET (empty diffs)

ℹ️ Generator diff

Generator Diff: vsdrops (html) vsdrops (raw diff) gist (raw diff) - Please review changes)

Pipeline on Agent
Hash: e5b4558abecc86b86bd4dee307942a78bd525c70 [PR build]

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2
Copy link
Collaborator

🔥 [CI Build #e5b4558] Test results 🔥

Test results

❌ Tests failed on VSTS: test results

0 tests crashed, 11 tests failed, 145 tests passed.

Failures

❌ monotouch tests (iOS) [attempt 2]

11 tests failed, 0 tests passed.

Failed tests

  • monotouch-test/iOS - simulator/Debug: Failed
  • monotouch-test/iOS - simulator/Debug (LinkSdk): Failed
  • monotouch-test/iOS - simulator/Debug (static registrar): Failed
  • monotouch-test/iOS - simulator/Release (all optimizations): Failed
  • monotouch-test/iOS - simulator/Debug (ARM64): Failed
  • monotouch-test/iOS - simulator/Release (NativeAOT, ARM64): Failed
  • monotouch-test/iOS - simulator/Debug (managed static registrar): Failed
  • monotouch-test/iOS - simulator/Release (managed static registrar, all optimizations): LaunchFailure
  • monotouch-test/iOS - simulator/Release (NativeAOT, x64): Crashed
  • monotouch-test/iOS - simulator/Debug (interpreter): Failed
  • monotouch-test/iOS - simulator/Release (interpreter): Failed

Html Report (VSDrops) Download

Successes

✅ cecil: All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (iOS): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (MacCatalyst): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (macOS): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (Multiple platforms): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (tvOS): All 1 tests passed. Html Report (VSDrops) Download
✅ framework: All 2 tests passed. Html Report (VSDrops) Download
✅ fsharp: All 4 tests passed. Html Report (VSDrops) Download
✅ generator: All 5 tests passed. Html Report (VSDrops) Download
✅ interdependent-binding-projects: All 4 tests passed. Html Report (VSDrops) Download
✅ introspection: All 6 tests passed. Html Report (VSDrops) Download
✅ linker: All 44 tests passed. Html Report (VSDrops) Download
✅ monotouch (MacCatalyst): All 15 tests passed. Html Report (VSDrops) Download
✅ monotouch (macOS): All 12 tests passed. Html Report (VSDrops) Download
✅ monotouch (tvOS): All 11 tests passed. Html Report (VSDrops) Download
✅ msbuild: All 2 tests passed. Html Report (VSDrops) Download
✅ sharpie: All 1 tests passed. Html Report (VSDrops) Download
✅ windows: All 3 tests passed. Html Report (VSDrops) Download
✅ xcframework: All 4 tests passed. Html Report (VSDrops) Download
✅ xtro: All 1 tests passed. Html Report (VSDrops) Download

macOS tests

✅ Tests on macOS Monterey (12): All 5 tests passed. Html Report (VSDrops) Download
✅ Tests on macOS Ventura (13): All 5 tests passed. [attempt 2] Html Report (VSDrops) Download
✅ Tests on macOS Sonoma (14): All 5 tests passed. Html Report (VSDrops) Download
✅ Tests on macOS Sequoia (15): All 5 tests passed. Html Report (VSDrops) Download
✅ Tests on macOS Tahoe (26): All 5 tests passed. Html Report (VSDrops) Download

Pipeline on Agent
Hash: e5b4558abecc86b86bd4dee307942a78bd525c70 [PR build]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants