Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Getting spammed with [CAMetalLayer nextDrawable] returning nil because device is nil on latest macOS Agent #6365

Closed
2 of 11 tasks
mrk-han opened this issue Oct 7, 2022 · 30 comments
Assignees
Labels
bug report investigate Collect additional information, like space on disk, other tool incompatibilities etc. OS: macOS

Comments

@mrk-han
Copy link

mrk-han commented Oct 7, 2022

Description

We are getting spammed by this one message in our emulator android test runs on the latest macOS agent.

You can see the same logs appear in this ticket too: #6252

I'm not sure anything is wrong because of it, but it is quite distracting to troubleshoot emulator issues with the spam.

[CAMetalLayer nextDrawable] returning nil because device is nil.

image

Platforms affected

  • Azure DevOps
  • GitHub Actions - Standard Runners
  • GitHub Actions - Larger Runners

Runner images affected

  • Ubuntu 18.04
  • Ubuntu 20.04
  • Ubuntu 22.04
  • macOS 10.15
  • macOS 11
  • macOS 12
  • Windows Server 2019
  • Windows Server 2022

Image version and build link

1.0.0.0-main-20220916-1

      - name: Run UI Tests
        uses: reactivecircus/android-emulator-runner@v2
        env:
          ANDROID_EMULATOR_WAIT_TIME_BEFORE_KILL: 120
        with:
          api-level: 30
          arch: x86
          target: google_atd
          profile: Nexus One
          emulator-options: -no-snapshot -noaudio -no-boot-anim -camera-back none
          disable-animations: true
          disk-size: 2048M
          script: ./gradlew connectedDebugAndroidTest

Is it regression?

Not sure

Expected behavior

I'm not quite sure why, but recently in our logs with the above emulator setup, we're getting spammed by this message [CAMetalLayer nextDrawable] returning nil because device is nil on latest macOS Agent, hundreds of times in a run

Actual behavior

Not getting this spam message

Repro steps

Start up an emulator on the latest (1.0.0.0-main-20220916-1) macOS agent with the above emulator architecture.

@mikhailkoliada mikhailkoliada added OS: Windows investigate Collect additional information, like space on disk, other tool incompatibilities etc. and removed needs triage labels Oct 7, 2022
@mikhailkoliada
Copy link
Contributor

@mrk-han thanks, we will take a look

@igorboskovic3
Copy link
Contributor

Hi @mrk-han can you provide exact repro steps?

@igorboskovic3
Copy link
Contributor

CC: @dsame

@mrk-han
Copy link
Author

mrk-han commented Oct 7, 2022

It's possible that the new screenshot APIs that come with android 3.5.0 espresso alpha could be causing this? But not sure. May be a red herring. Check below for the screenshot sample with that setup and GitHub action stuff. I'd need to fork that and add the GitHub action .yml that I pasted below, but not sure I have time for that today.

/**
 * An internal variant of [takeScreenshot] that skips an idle sync call.
 *
 * This intended for failure handling cases where caller does not want to wait for main thread to be
 * idle.
 *
 * @return a [Bitmap]
 * @throws [IllegalStateException] if called on the main thread. This is a limitation of connecting
 * to UiAutomation, [RuntimeException] if UiAutomation fails to take the screenshot
 *
 * @hide
 */
@ExperimentalTestApi
@RestrictTo(RestrictTo.Scope.LIBRARY_GROUP)
@RequiresApi(Build.VERSION_CODES.JELLY_BEAN_MR2)
@Suppress("FutureReturnValueIgnored")
@Throws(RuntimeException::class)
fun takeScreenshotNoSync(): Bitmap {
  Checks.checkState(Looper.myLooper() != Looper.getMainLooper())

  val bitmapFuture: ResolvableFuture<Bitmap> = ResolvableFuture.create()
  val mainExecutor = HandlerExecutor(Handler(Looper.getMainLooper()))
  val uiAutomation = getInstrumentation().uiAutomation
  if (uiAutomation == null) {
    throw RuntimeException("uiautomation is null")
  }

  if (!HardwareRendererCompat.isDrawingEnabled()) {
    HardwareRendererCompat.setDrawingEnabled(true)
    bitmapFuture.addListener({ HardwareRendererCompat.setDrawingEnabled(false) }, mainExecutor)
  }

  try {
    forceRedrawGlobalWindowViews(mainExecutor).get(5, TimeUnit.SECONDS)
  } catch (e: Exception) {
    Log.w("takeScreenshot", "force redraw failed. Proceeding with screenshot", e)
  }

  // take the screenshot on the next frame to increase probability the draw from previous step is
  // committed
  mainExecutor.execute {
    Choreographer.getInstance().postFrameCallback {
      val bitmap = uiAutomation.takeScreenshot()
      if (bitmap == null) {
        bitmapFuture.setException(RuntimeException("uiAutomation.takeScreenshot returned null"))
      } else {
        bitmapFuture.set(bitmap)
      }
    }
  }

You can find code to use here: https://github.com/android/testing-samples/tree/main/ui/espresso/ScreenshotSample but I think we need to hook this sample up to a basic Github Action using the following:


name: UI Testing

on:
  push:
    branches:
      - develop
  workflow_dispatch:

jobs:
  ui-testing:
    runs-on: macos-latest
    environment: Debug

    steps:
      - name: checkout
        uses: actions/checkout@v3
      - name: Set Up JDK
        uses: actions/setup-java@v1
        with:
          java-version: 11

      - name: Setup Android SDK
        uses: android-actions/setup-android@v2

      - name: Run UI Tests
        uses: reactivecircus/android-emulator-runner@v2
        env:
          ANDROID_EMULATOR_WAIT_TIME_BEFORE_KILL: 120
        with:
          api-level: 30
          arch: x86
          target: google_atd
          profile: Nexus One
          emulator-options: -no-snapshot -noaudio -no-boot-anim -camera-back none
          disable-animations: true
          disk-size: 2048M
          script: ./gradlew connectedDebugAndroidTest

      - name: Archive Index.html
        uses: actions/upload-artifact@v3
        continue-on-error: true
        if: success() || failure() # always() will cause the function to run even if the build is canceled, so it is best to use success || failure
        with:
          name: Index.html Run Number=${{ github.run_number }}
          path: |
            /app/build/reports/androidTests/**/*.html
          retention-days: 30

      - name: Archive test-result.pb
        uses: actions/upload-artifact@v3
        continue-on-error: true
        if: success() || failure()
        with:
          name: Test-result.pb Run Number=${{ github.run_number }}
          path: |
            /app/**/*.pb
          retention-days: 30

      - name: Archive Build and Outputs Folder (screenshots + logs)
        uses: actions/upload-artifact@v3
        continue-on-error: true
        if: success() || failure()
        with:
          name: Run Number=${{ github.run_number }}
          path: |
            **/app/build/reports/**
            **/app/build/outputs/**
            !**/app/build/outputs/apk/**
            !**/app/build/outputs/code_coverage/**
            !**/app/build/outputs/managed_device_code_coverage/**
            **/app/build/snapshot/**
            **/app/build/generated/crashlytics/**
          retention-days: 30

      - name: Check for HAXM installation
        run: emulator -accel-check && emulator -accel-check > ${{ github.workspace }}/haxm-check-${{ github.run_number }}.txt

      - name: Archive Haxm Check
        uses: actions/upload-artifact@v3
        continue-on-error: true
        if: success() || failure()
        with:
          name: emulator-haxm-check
          path: ${{ github.workspace }}/emulator.txt

@mrk-han
Copy link
Author

mrk-han commented Oct 7, 2022

image

Example of some of the spam

@mrk-han
Copy link
Author

mrk-han commented Oct 7, 2022

@igorboskovic3 @mikhailkoliada

I think the screenshot API is a red herring. I noticed in my log it starts appearing after we revoke the microphone permission

/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[209](*/actions/runs/3206368549/jobs/5240032010#step:11:209)
adb: device offline
[210](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:210)
The process '/Users/runner/Library/Android/sdk/platform-tools/adb' failed with exit code 1
[211](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:211)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[212](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:212)
The process '/Users/runner/Library/Android/sdk/platform-tools/adb' failed with exit code 1
[213](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:213)
adb: device offline
[214](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:214)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[215](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:215)
adb: device offline
[216](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:216)
The process '/Users/runner/Library/Android/sdk/platform-tools/adb' failed with exit code 1
[217](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:217)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[218](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:218)
The process '/Users/runner/Library/Android/sdk/platform-tools/adb' failed with exit code 1
[219](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:219)
adb: device offline
[220](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:220)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[221](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:221)
The process '/Users/runner/Library/Android/sdk/platform-tools/adb' failed with exit code 1
[222](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:222)
adb: device offline
[223](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:223)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[224](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:224)
The process '/Users/runner/Library/Android/sdk/platform-tools/adb' failed with exit code 1
[225](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:225)
adb: device offline
[226](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:226)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[227](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:227)
The process '/Users/runner/Library/Android/sdk/platform-tools/adb' failed with exit code 1
[228](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:228)
adb: device offline
[229](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:229)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[231](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:231)
WARNING | /etc/localtime does not point to zoneinfo-compatible timezone name
[232](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:232)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[234](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:234)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[236](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:236)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[238](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:238)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[240](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:240)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[242](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:242)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[244](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:244)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[246](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:246)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[248](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:248)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[250](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:250)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[252](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:252)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[254](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:254)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[256](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:256)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[258](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:258)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[260](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:260)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[262](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:262)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[264](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:264)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[266](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:266)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[268](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:268)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[270](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:270)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[272](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:272)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[274](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:274)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[276](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:276)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[278](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:278)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[280](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:280)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[282](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:282)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[284](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:284)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[286](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:286)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[288](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:288)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[290](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:290)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[292](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:292)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[294](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:294)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[296](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:296)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[298](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:298)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[300](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:300)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[302](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:302)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[304](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:304)
/Users/runner/Library/Android/sdk/platform-tools/adb shell getprop sys.boot_completed
[305](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:305)
1
[306](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:306)
Emulator booted.
[307](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:307)
/Users/runner/Library/Android/sdk/platform-tools/adb shell input keyevent 82
[308](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:308)
Disabling animations.
[309](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:309)
/Users/runner/Library/Android/sdk/platform-tools/adb shell settings put global window_animation_scale 0.0
[310](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:310)
/Users/runner/Library/Android/sdk/platform-tools/adb shell settings put global transition_animation_scale 0.0
[311](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:311)
/Users/runner/Library/Android/sdk/platform-tools/adb shell settings put global animator_duration_scale 0.0
[312](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:312)

[313](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:313)
/bin/sh -c ./gradlew connectedDebugAndroidTest
[314](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:314)
Downloading https://services.gradle.org/distributions/gradle-7.5.1-bin.zip
[315](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:315)
WARNING | boot completed
[316](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:316)
INFO    | boot completed
[317](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:317)
INFO    | boot time 160699 ms
[318](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:318)
INFO    | Increasing screen off timeout, logcat buffer size to 2M.
[319](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:319)
INFO    | Revoking microphone permissions for Google App.
[320](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:320)
...........10%............20%.......2022-10-07 16:57:44.500 qemu-system-x86_64[2506:11383] [CAMetalLayer nextDrawable] returning nil because device is nil.
[321](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:321)
..2022-10-07 16:57:44.654 qemu-system-x86_64[2506:11383] [CAMetalLayer nextDrawable] returning nil because device is nil.
[322](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:322)
..2022-10-07 16:57:44.813 qemu-system-x86_64[2506:11383] [CAMetalLayer nextDrawable] returning nil because device is nil.
[323](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:323)
30%............40%...........50%............60%...........70%..2022-10-07 16:57:49.061 qemu-system-x86_64[2506:11383] [CAMetalLayer nextDrawable] returning nil because device is nil.
[324](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:324)
..........80%...........90%............100%
[325](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:325)
2022-10-07 16:57:49.769 qemu-system-x86_64[2506:11383] [CAMetalLayer nextDrawable] returning nil because device is nil.
[326](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:326)
2022-10-07 16:57:50.255 qemu-system-x86_64[2506:11383] [CAMetalLayer nextDrawable] returning nil because device is nil.
[327](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:327)
2022-10-07 16:57:50.384 qemu-system-x86_64[2506:11383] [CAMetalLayer nextDrawable] returning nil because device is nil.
[328](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:328)
2022-10-07 16:57:50.492 qemu-system-x86_64[2506:11383] [CAMetalLayer nextDrawable] returning nil because device is nil.
[329](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:329)
2022-10-07 16:57:50.543 qemu-system-x86_64[2506:11383] [CAMetalLayer nextDrawable] returning nil because device is nil.
[330](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:330)
2022-10-07 16:57:50.668 qemu-system-x86_64[2506:11383] [CAMetalLayer nextDrawable] returning nil because device is nil.
[331](https://github.com/redacted-org/redacted-app-name//actions/runs/3206368549/jobs/5240032010#step:11:331)
Starting a Gradle Daemon (subsequent builds will be faster)

@mrk-han
Copy link
Author

mrk-han commented Oct 7, 2022

Nevermind, it seems the -no-audio flag was a red herring as well.

@dsame dsame self-assigned this Oct 10, 2022
@mrk-han
Copy link
Author

mrk-han commented Oct 12, 2022

image

I ran our pipeline on a variety of system-images and emulators, and they all had the same spam.

I also ran an emulator -accel-check and learned that HAXM is not enabled on these agents by default, which is not what I had originally heard about these macOS agents.

@igorboskovic3
Copy link
Contributor

igorboskovic3 commented Oct 12, 2022

Hi @mrk-han, can you check this thread and use this snippet from that thread to resolve your issue?

@igorboskovic3
Copy link
Contributor

Yes, HAXM isnt installed on our MacOS images, you can check same thread for installation, thank you

@mrk-han
Copy link
Author

mrk-han commented Oct 12, 2022

I'll implement shortly and let you know. Thank you!

@ssrinivasKG
Copy link

@mrk-han did installing HAXM fix the issue for you?

@mrk-han
Copy link
Author

mrk-han commented Oct 18, 2022

@ssrinivasKG @igorboskovic3

I attempted to install HAXM like this:

      - name: Install Haxm
        run: brew install intel-haxm

It gave the following output:

2022-10-17T20:19:47.3593370Z If the installation fails, retry after you enable it in:
2022-10-17T20:19:47.3594550Z   System Preferences → Security & Privacy → General
2022-10-17T20:19:47.3595020Z 
2022-10-17T20:19:47.3595820Z For more information, refer to vendor documentation or this Apple Technical Note:
2022-10-17T20:19:47.3598960Z   �[4mhttps://developer.apple.com/library/content/technotes/tn2459/_index.html�[24m
2022-10-17T20:19:47.3599510Z 
2022-10-17T20:19:47.5647050Z �[34m==>�[0m �[1mDownloading https://github.com/intel/haxm/releases/download/v7.7.1/haxm-macosx_v7_7_1.zip�[0m
2022-10-17T20:19:47.5671010Z �[34m==>�[0m �[1mDownloading from https://objects.githubusercontent.com/github-production-release-asset-2e65be/98257850/f4992069-e192-4393-ae91-8018447bbd8f?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20221017%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20221017T201947Z&X-Amz-Expires=300&X-Amz-Signature=452a05b62ae34459d99876bfaa178f80e229d5a08772f70e9e188540ece11ce9&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=98257850&response-content-disposition=attachment%3B%20filename%3Dhaxm-macosx_v7_7_1.zip&response-content-type=application%2Foctet-stream�[0m
2022-10-17T20:19:55.7430210Z �[33mWarning:�[0m macOS's Gatekeeper has been disabled for this Cask
2022-10-17T20:19:55.7431410Z �[32m==>�[0m �[1mInstalling Cask �[32mintel-haxm�[39m�[0m
2022-10-17T20:19:55.8254240Z �[34m==>�[0m �[1mRunning installer script 'silent_install.sh'�[0m
2022-10-17T20:20:32.2334970Z Silent installation Pass! 
2022-10-17T20:20:32.3790420Z 🍺  intel-haxm was successfully installed!

and to verify, after the tests I ran:

      - name: Check for HAXM installation
        if: always()
        run: |
          emulator -accel-check 2>&1 | tee ${{ github.workspace }}/haxm-check-${{ github.run_number }}.txt

but received:

2022-10-17T20:48:28.5246820Z accel:
2022-10-17T20:48:28.6775500Z 8
2022-10-17T20:48:28.6778470Z HAXM is not installed on this machine (/dev/HAX is missing).

And then the bug still existed:

2022-10-17T20:24:29.9441900Z INFO    | Revoking microphone permissions for Google App.
2022-10-17T20:24:30.6995530Z 2022-10-17 20:24:30.699 qemu-system-x86_64[4667:17550] [CAMetalLayer nextDrawable] returning nil because device is nil.
2022-10-17T20:24:30.7847920Z 2022-10-17 20:24:30.775 qemu-system-x86_64[4667:17550] [CAMetalLayer nextDrawable] returning nil because device is nil.
2022-10-17T20:24:30.8452300Z 2022-10-17 20:24:30.836 qemu-system-x86_64[4667:17550] [CAMetalLayer nextDrawable] returning nil because device is nil.
2022-10-17T20:24:30.9550040Z 2022-10-17 20:24:30.949 qemu-system-x86_64[4667:17550] [CAMetalLayer nextDrawable] returning nil because device is nil.
2022-10-17T20:24:31.1306670Z 2022-10-17 20:24:31.130 qemu-system-x86_64[4667:17550] [CAMetalLayer nextDrawable] returning nil because device is nil.
2022-10-17T20:24:31.2130740Z 2022-10-17 20:24:31.207 qemu-system-x86_64[4667:17550] [CAMetalLayer nextDrawable] returning nil because device is nil.
2022-10-17T20:24:31.2506280Z 2022-10-17 20:24:31.246 qemu-system-x86_64[4667:17550] [CAMetalLayer nextDrawable] returning nil because device is nil.
2022-10-17T20:24:31.3497000Z 2022-10-17 20:24:31.348 qemu-system-x86_64[4667:17550] [CAMetalLayer nextDrawable] returning nil because device is nil.
2022-10-17T20:24:32.9928510Z 2022-10-17 20:24:32.992 qemu-system-x86_64[4667:17550] [CAMetalLayer nextDrawable] returning nil because device is nil.
2022-10-17T20:24:33.1078150Z 2022-10-17 20:24:33.106 qemu-system-x86_64[4667:17550] [CAMetalLayer nextDrawable] returning nil because device is nil.
2022-10-17T20:24:33.1397840Z 2022-10-17 20:24:33.139 qemu-system-x86_64[4667:17550] [CAMetalLayer nextDrawable] returning nil because device is nil.
2022-10-17T20:24:33.2133210Z 2022-10-17 20:24:33.212 qemu-system-x86_64[4667:17550] [CAMetalLayer nextDrawable] returning nil because device is nil.
2022-10-17T20:24:33.2549300Z 2022-10-17 20:24:33.254 qemu-system-x86_64[4667:17550] [CAMetalLayer nextDrawable] returning nil because device is nil.
2022-10-17T20:24:33.3643330Z 2022-10-17 20:24:33.363 qemu-system-x86_64[4667:17550] [CAMetalLayer nextDrawable] returning nil because device is nil.
2022-10-17T20:24:33.5150420Z 2022-10-17 20:24:33.514 qemu-system-x86_64[4667:17550] [CAMetalLayer nextDrawable] returning nil because device is nil.
2022-10-17T20:24:33.5529410Z 2022-10-17 20:24:33.552 qemu-system-x86_64[4667:17550] [CAMetalLayer nextDrawable] returning nil because device is nil.
2022-10-17T20:24:33.6582700Z 2022-10-17 20:24:33.648 qemu-system-x86_64[4667:17550] [CAMetalLayer nextDrawable] returning nil because device is nil.
2022-10-17T20:24:33.7710100Z 2022-10-17 20:24:33.762 qemu-system-x86_64[4667:17550] [CAMetalLayer nextDrawable] returning nil because device is nil.
2022-10-17T20:24:33.8576420Z 2022-10-17 20:24:33.849 qemu-system-x86_64[4667:17550] [CAMetalLayer nextDrawable] returning nil because device is nil.
2022-10-17T20:24:33.9728790Z 2022-10-17 20:24:33.972 qemu-system-x86_64[4667:17550] [CAMetalLayer nextDrawable] returning nil because device is nil.
2022-10-17T20:24:34.0713870Z 2022-10-17 20:24:34.070 qemu-system-x86_64[4667:17550] [CAMetalLayer nextDrawable] returning nil because device is nil.
2022-10-17T20:24:34.2971050Z ##[endgroup]
2022-10-17T20:24:34.3073950Z [command]/bin/sh -c echo "Generated AVD snapshot for caching."

@mrk-han
Copy link
Author

mrk-han commented Oct 18, 2022

Note: This is also showing in the logs from @igorboskovic3 's run here: https://github.com/igorboskovic3/actions-test/actions/runs/3023933068/jobs/4864768035

@ssrinivasKG
Copy link

Based on the comments on my issue, does not seem like something fixable #6369

@igorboskovic3
Copy link
Contributor

igorboskovic3 commented Oct 19, 2022

@mrk-han this is my testing repo, i wouldnt use that as a reference, as I answered on other thread here, you have there successful and unsuccessful actions so you should use for example successful one https://github.com/igorboskovic3/actions-test/actions/runs/3239143575. Please use this snippet or android-27 or below for successful action

@mrk-han
Copy link
Author

mrk-han commented Oct 26, 2022

@igorboskovic3 Thank you so much. I got HAXM working and I am still seeing this output spammed

@igorboskovic3
Copy link
Contributor

@mrk-han Code is working and you have output spammed or doesnt work at all?

@mrk-han
Copy link
Author

mrk-han commented Oct 27, 2022

@igorboskovic3 Well, the stability of the emulator itself is causing UI tests to be very unstable. That is the heart of the issue. When debugging things, this spam is incredible excessive (happening anywhere from 3-7 times a second), it makes it hard to read.

My concern is the amount of times this error is happening is either indicative of, or itself causing performance issues. Or means something is misconfigured.

Here is a picture of the state of the machine when these logs are getting spammed:
image

And this is happening on every API level / Architecture I run tests with

@mrk-han
Copy link
Author

mrk-han commented Oct 27, 2022

I am really not sure if this is a problem because of the Google Emulator team, or the MacOS Runner configuration.

@ssrinivasKG
Copy link

Looking at the screenshot, looks very much like a GitHub runner configuration issue. Especially the system dialogue complaining about nested virtualization and plus the GPU driver information system image as well

@mrk-han
Copy link
Author

mrk-han commented Nov 1, 2022

@ssrinivasKG Exactly, these macOS agents are VMs, and the Android Emulator is a VM -- so it is using Nested Virtualization. But without proper Hardware Support, the Emulators are super flakey -- which is why we are trying to enable HAXM to help.

@igorboskovic3 Across many teams, plus others I have spoken to, none of us are able to get reliable test suites (with Android Emulators) working on Github's MacOS Runners.

@kopax-polyconseil
Copy link

kopax-polyconseil commented Nov 2, 2022

Hello, we have the exact same issue, this is our branch : pass-culture/pass-culture-app-native#3840

image

image

We are trying to run appium to perform e2e tests, did you find a workaround ? I have tried all github action for avd and none works.

@igorboskovic3
Copy link
Contributor

@ssrinivasKG Exactly, these macOS agents are VMs, and the Android Emulator is a VM -- so it is using Nested Virtualization. But without proper Hardware Support, the Emulators are super flakey -- which is why we are trying to enable HAXM to help.

@igorboskovic3 Across many teams, plus others I have spoken to, none of us are able to get reliable test suites (with Android Emulators) working on Github's MacOS Runners.

Yes, we are also tried various approaches, but we dont have of support of HAXM for VMs from one side (as you can see here) and also we get various results randomly for same code which suggest it could be performance issue.

@voghDev
Copy link

voghDev commented Dec 2, 2022

I was getting spammed with this message as well as getting a grey/black screen in my Android emulator, and I could fix it by changing the rendering mode.
This was the way I was launching the emulator

echo "no" | $ANDROID_HOME/emulator/emulator "-avd" "Nexus_5X_API_26" "-no-audio" "-no-boot-anim" "-gpu" "swiftshader_indirect" &

and after removing the -gpu swiftshader_indirect part, it worked again:

echo "no" | $ANDROID_HOME/emulator/emulator "-avd" "Nexus_5X_API_26" "-no-audio" "-no-boot-anim" &

This may not be the case of everyone, but I wanted to post it as it can be helpful 🙂
Other way of changing the rendering mode is going to Settings (...) in the emulator, then Settings again, and Advanced. There you have the OpenGL ES renderer and the OpenGL ES API level. I suggest to try all possible values. Important: Don't forget to shutdown and restart the emulator after each value you try. Else you won't see any change.
A little more info about launching the emulator with different accelerations

@mrk-han
Copy link
Author

mrk-han commented Dec 2, 2022

@voghDev The problem is we have to use swiftshader_indirect on Github's macOS runners. This is good info though, thank you.

@mrk-han
Copy link
Author

mrk-han commented Dec 2, 2022

@ssrinivasKG Exactly, these macOS agents are VMs, and the Android Emulator is a VM -- so it is using Nested Virtualization. But without proper Hardware Support, the Emulators are super flakey -- which is why we are trying to enable HAXM to help.
@igorboskovic3 Across many teams, plus others I have spoken to, none of us are able to get reliable test suites (with Android Emulators) working on Github's MacOS Runners.

Yes, we are also tried various approaches, but we dont have of support of HAXM for VMs from one side (as you can see here) and also we get various results randomly for same code which suggest it could be performance issue.

It turns out, we don't need HAXM anymore -- because of the default hypervisor provided by Apple. However, this is only VM acceleration and not Graphics acceleration.

If you use the newer Android tooling, then emulator -accel-check passes because it leverages this new native hypervisor and does not need HAXM anymore. $ANDROID_HOME/emulator/emulator

@ssrinivasKG
Copy link

@mrk-han Have you got a working emulator now? We're running out of options for our E2E tests

@dsame
Copy link
Contributor

dsame commented Mar 2, 2023

The issue caused by the low performance of the runner hosts it is reproduced locally it and does not depends on the software configuration. It is expected to be solved with the hardware upgrade.

@jhonatan-3a
Copy link

so this is about performance in the runners ? im using macos11, if use macos13 will it go away or how does it solveS?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug report investigate Collect additional information, like space on disk, other tool incompatibilities etc. OS: macOS
Projects
None yet
Development

No branches or pull requests

9 participants