Skip to content

E2E: verify audio stream is changing (not silent) #755

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 3, 2025
Merged

Conversation

mjh1
Copy link
Contributor

@mjh1 mjh1 commented Jun 2, 2025

PR Type

Tests, Enhancement


Description

  • Add fake audio capture flag in Playwright config

  • Implement audio-diff assertion helper in common.ts

  • Integrate audio checks in video.spec.ts tests

  • Expose broadcast-mute test ID in component


Changes walkthrough 📝

Relevant files
Configuration changes
playwright.config.ts
Enable fake audio capture option                                                 

apps/app-e2e/playwright.config.ts

  • Added --use-file-for-fake-audio-capture=./test-audio.wav
  • Configured Chrome launch options for audio testing
  • +1/-0     
    Tests
    common.ts
    Add audio diff assertion helper                                                   

    apps/app-e2e/tests/common.ts

  • Imported Page and defined MIN_AUDIO_DIFF
  • Created assertAudioChanging helper to measure audio diff
  • Logs max audio change percentage
  • +50/-1   
    video.spec.ts
    Integrate audio checks and logger                                               

    apps/app-e2e/tests/video.spec.ts

  • Imported and used assertAudioChanging
  • Switched to createTestLogger for logging
  • Replaced audioTracks assertions with mute toggle logic
  • Added 10s stream sleep after checks
  • +14/-21 
    broadcast.tsx
    Expose broadcast mute test ID                                                       

    apps/app/components/playground/broadcast.tsx

  • Added data-testid="broadcast-mute" to audio trigger
  • Enables E2E test to locate mute button
  • +4/-1     
    Additional files
    test-audio.wav [link]   

    Need help?
  • Type /help how to ... in the comments thread for any questions about PR-Agent usage.
  • Check out the documentation for more information.
  • Copy link

    vercel bot commented Jun 2, 2025

    The latest updates on your projects. Learn more about Vercel for Git ↗︎

    Name Status Preview Comments Updated (UTC)
    pipelines-app ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jun 3, 2025 3:30pm

    Copy link

    github-actions bot commented Jun 2, 2025

    PR Reviewer Guide 🔍

    (Review updated until commit d5e2795)

    Here are some key observations to aid the review process:

    ⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
    🧪 PR contains tests
    🔒 No security concerns identified
    ⚡ Recommended focus areas for review

    Missing Import

    createTestLogger is used but not imported, which will cause runtime errors.

    const logger = createTestLogger(test.info());
    Missing Assertion

    assertAudioChanging has its expect assertion commented out, so the test does not fail when audio change is below threshold.

      // Uncomment once the broadcast-mute testid attribute change is on prod
      // expect(
      //   maxChange,
      //   `Frames changed only ${(maxChange * 100).toFixed(2)}%, should exceed ${(MIN_AUDIO_DIFF * 100).toFixed(2)}%`,
      // ).toBeGreaterThan(MIN_AUDIO_DIFF);
    }
    AudioContext Leak

    An AudioContext is created in the page context but never closed, which may lead to resource leaks during tests.

    const context = new AudioContext();
    const stream = video.srcObject as MediaStream;
    if (!stream || stream.getAudioTracks().length === 0) return -1;
    
    const analyser = context.createAnalyser();
    const source = context.createMediaStreamSource(stream);
    source.connect(analyser);

    Copy link

    github-actions bot commented Jun 2, 2025

    PR Code Suggestions ✨

    Latest suggestions up to d5e2795
    Explore these optional code suggestions:

    CategorySuggestion                                                                                                                                    Impact
    Possible issue
    Fail on missing audio stream

    Instead of returning a sentinel on missing audio, fail fast so tests clearly
    indicate missing audio tracks. Throwing will immediately surface the absence of an
    audio stream.

    apps/app-e2e/tests/common.ts [338]

    -if (!stream || stream.getAudioTracks().length === 0) return -1;
    +if (!stream || stream.getAudioTracks().length === 0) {
    +  throw new Error("No audio tracks found on the video element");
    +}
    Suggestion importance[1-10]: 7

    __

    Why: The test currently returns -1 silently when no audio tracks are found; throwing an error surfaces missing audio streams immediately and avoids false positives.

    Medium
    General
    Close AudioContext after use

    Close the AudioContext before returning to avoid leaking audio resources in the
    browser. This ensures the test doesn’t hold onto system audio handles.

    apps/app-e2e/tests/common.ts [336-370]

     const context = new AudioContext();
     const stream = video.srcObject as MediaStream;
     ...
    -return maxChange;
    +const result = maxChange;
    +context.close();
    +return result;
    Suggestion importance[1-10]: 3

    __

    Why: Closing the AudioContext helps prevent resource leaks but has minimal impact on test correctness or functionality.

    Low

    Previous suggestions

    Suggestions up to commit 69e5c5f
    CategorySuggestion                                                                                                                                    Impact
    General
    Enable fake audio capture

    Uncomment and provide the fake audio capture argument so that the browser uses a
    predictable audio source during E2E tests. Use a resolved path to ensure it works
    across environments.

    apps/app-e2e/playwright.config.ts [60]

    -// "--use-file-for-fake-audio-capture=./test-audio.wav",
    +import path from "path";
    +...
    +args: [
    +  "--disable-web-security",
    +  "--use-fake-device-for-media-stream",
    +  `--use-file-for-fake-audio-capture=${path.resolve(__dirname, "test-audio.wav")}`,
    +],
    Suggestion importance[1-10]: 7

    __

    Why: Uncommenting --use-file-for-fake-audio-capture with path.resolve improves test determinism by supplying a consistent audio source.

    Medium
    Limit waveform logging loop

    Replace the infinite recursive timeout with a controlled interval and clear it after
    a few samples to prevent leaking timers in the page.

    apps/app-e2e/tests/video.spec.ts [156-168]

    -function logWaveform() {
    +const interval = setInterval(() => {
       analyser.getByteTimeDomainData(dataArray);
    -  // Print a basic snapshot of the waveform (min/max)
       const min = Math.min(...dataArray);
       const max = Math.max(...dataArray);
    -  console.log(
    -    `Waveform: min=${min}, max=${max}, mid=${dataArray[Math.floor(bufferLength / 2)]}`,
    -  );
    -  // Repeat every 500ms
    -  setTimeout(logWaveform, 500);
    -}
    +  console.log(`Waveform: min=${min}, max=${max}, mid=${dataArray[Math.floor(bufferLength / 2)]}`);
    +}, 500);
    +setTimeout(() => clearInterval(interval), 2000);
    Suggestion importance[1-10]: 7

    __

    Why: Switching from recursive setTimeout to a bounded setInterval with clearInterval prevents timer leaks in the page.

    Medium
    Reduce wait timeout

    Reduce the timeout back to a shorter, reasonable duration (e.g., 10 seconds) to
    avoid excessively long test runs.

    apps/app-e2e/tests/video.spec.ts [197]

    -await page.waitForTimeout(100 * 1000);
    +await page.waitForTimeout(10 * 1000);
    Suggestion importance[1-10]: 6

    __

    Why: Reducing page.waitForTimeout lowers overall test duration and avoids excessively long idle waits.

    Low
    Possible issue
    Throw on missing media stream

    Fail the test immediately if no MediaStream is available inside the browser context
    instead of silently logging. Throwing an error in the page context bubbles up to the
    test runner.

    apps/app-e2e/tests/video.spec.ts [143-145]

     if (!(stream instanceof MediaStream)) {
    -  console.log("No stream found");
    -  return;
    +  throw new Error("No MediaStream found for audio analysis");
     }
    Suggestion importance[1-10]: 6

    __

    Why: Failing fast with an error instead of logging ensures missing MediaStream is treated as a test failure rather than silently passing.

    Low

    @mjh1 mjh1 temporarily deployed to Preview - e2e June 3, 2025 11:38 — with GitHub Actions Inactive
    @mjh1 mjh1 requested review from junhyr and leszko June 3, 2025 11:38
    @mjh1 mjh1 marked this pull request as ready for review June 3, 2025 11:38
    Copy link

    github-actions bot commented Jun 3, 2025

    Persistent review updated to latest commit d5e2795

    @mjh1 mjh1 changed the title Audio E2E checks E2E: verify audio stream is changing (not silent) Jun 3, 2025
    @mjh1 mjh1 temporarily deployed to Preview - e2e June 3, 2025 15:28 — with GitHub Actions Inactive
    @mjh1 mjh1 merged commit 24ea098 into main Jun 3, 2025
    6 checks passed
    @mjh1 mjh1 deleted the mh/audio-e2e branch June 3, 2025 15:31
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    2 participants