Version
Environment
@midscene/core: 1.10.3
@midscene/web: 1.10.3
Playwright: 1.48.0
Node.js: 22.23.1
OS: Ubuntu 22.04 LTS (headless, no GPU)
Browser: Chromium (headless mode)
MIDSCENE_OUTPUT_FORMAT: single-html
Details
Describe the bug
When running Midscene.js with Playwright in a headless Linux environment (no GPU), the AI agent correctly identifies and interacts with page elements, but the screenshots saved in the HTML report are completely black.
Environment
@midscene/core: 1.10.3
@midscene/web: 1.10.3
- Playwright: 1.48.0
- Node.js: 22.23.1
- OS: Ubuntu 22.04 LTS (headless, no GPU)
- Browser: Chromium (headless mode)
MIDSCENE_OUTPUT_FORMAT: single-html
Steps to Reproduce
- Run a Midscene script on a Linux server with headless Chromium
- The target page has CSS animations and gradient backgrounds (a login page with animated background)
- Script uses
agent.aiTap(), agent.aiAssert() etc.
- All AI actions succeed — the agent correctly locates buttons, input fields, and verifies page content
- Open the generated HTML report
Expected Behavior
The HTML report should show screenshots of the page at each step, matching what the AI "saw" during execution.
Actual Behavior
All screenshots in the report are completely black. The HTML file is 3-4MB (screenshots ARE embedded as base64 in single-html mode), but the image content is black.
Key Observation
The AI agent can correctly identify elements on the page (e.g., "账号输入框" at coordinates [1330, 346], "登录按钮", etc.). This means the screenshots taken for AI processing are rendered correctly. However, the screenshots saved to the report are black.
This suggests that the screenshots used for AI inference and the screenshots saved to the report are captured at different times or through different mechanisms, and the report screenshots fail to render properly.
What I've Tried
--disable-gpu flag → screenshots still black
--use-angle=swiftshader flag → screenshots completely broken (file not found)
MIDSCENE_OUTPUT_FORMAT=single-html → screenshots embedded but still black
MIDSCENE_OUTPUT_FORMAT=html-and-external-assets → external PNG files not generated at all
- Adding
waitForTimeout(3000) before actions → no effect on report screenshots
Script Example
page.setViewportSize({ width: 1600, height: 900 });
await page.goto('xxxxx');
await page.waitForTimeout(3000);
await page.getByText('账号密码登录').click();
await page.waitForTimeout(1000);
await agent.aiTap('账号输入框');
await page.keyboard.type('xxx');
await agent.aiTap('密码输入框');
await page.keyboard.type('password');
await agent.aiTap('登录按钮');
await page.waitForTimeout(3000);
### Reproduce link
N/A
### Reproduce Steps
1. Set up a Linux server (Ubuntu 22.04) with Node.js 22+ and no GPU
2. Install Chromium via Playwright: npx playwright install chromium
3. Install Midscene: npm install @midscene/core @midscene/web
4. Create a script that uses PlaywrightAgent to interact with a page that has CSS animations/gradients:
const { chromium } = require('playwright');
const { PlaywrightAgent } = require('@midscene/web/playwright');
process.env.MIDSCENE_OUTPUT_FORMAT = 'single-html';
process.env.MIDSCENE_MODEL_NAME = 'your-model';
process.env.MIDSCENE_MODEL_API_KEY = 'your-key';
process.env.MIDSCENE_MODEL_FAMILY = 'your-family';
(async () => {
const browser = await chromium.launch({ headless: true, args: ['--no-sandbox'] });
const page = await browser.newPage({ viewport: { width: 1600, height: 900 } });
const agent = new PlaywrightAgent(page);
await page.goto('xxxx');
await page.waitForTimeout(3000);
await agent.aiTap('账号输入框');
await page.keyboard.type('test');
// Check the generated HTML report - screenshots will be black
await browser.close();
})();
5. Open the generated HTML report in a browser
6. Observe that all screenshots in the report are completely black, even though AI correctly identified elements
Version
Details
Describe the bug
When running Midscene.js with Playwright in a headless Linux environment (no GPU), the AI agent correctly identifies and interacts with page elements, but the screenshots saved in the HTML report are completely black.
Environment
@midscene/core: 1.10.3@midscene/web: 1.10.3MIDSCENE_OUTPUT_FORMAT:single-htmlSteps to Reproduce
agent.aiTap(),agent.aiAssert()etc.Expected Behavior
The HTML report should show screenshots of the page at each step, matching what the AI "saw" during execution.
Actual Behavior
All screenshots in the report are completely black. The HTML file is 3-4MB (screenshots ARE embedded as base64 in
single-htmlmode), but the image content is black.Key Observation
The AI agent can correctly identify elements on the page (e.g., "账号输入框" at coordinates [1330, 346], "登录按钮", etc.). This means the screenshots taken for AI processing are rendered correctly. However, the screenshots saved to the report are black.
This suggests that the screenshots used for AI inference and the screenshots saved to the report are captured at different times or through different mechanisms, and the report screenshots fail to render properly.
What I've Tried
--disable-gpuflag → screenshots still black--use-angle=swiftshaderflag → screenshots completely broken (file not found)MIDSCENE_OUTPUT_FORMAT=single-html→ screenshots embedded but still blackMIDSCENE_OUTPUT_FORMAT=html-and-external-assets→ external PNG files not generated at allwaitForTimeout(3000)before actions → no effect on report screenshotsScript Example