Skip to content

[Bug]: Element positioning exceeds the screen size #2929

Description

@Richard1ybb

Version

System:
    OS: macOS 14.1.1
    CPU: (8) arm64 Apple M2
    Memory: 135.94 MB / 16.00 GB
    Shell: 5.9 - /bin/zsh
  Browsers:
    Chrome: 150.0.7871.188
    Safari: 17.1

Details

task runner is in error state, cannot append task
error=ServiceError: failed to locate element:
Failed to parse locate result: locate result coordinates [62,3268,698,3438] exceed image size [0, 1440]x[0, 3200]. shape=bbox order=xy limits=[1440,3200,1440,3200]
ServiceError: failed to locate element:
Failed to parse locate result: locate result coordinates [62,3268,698,3438] exceed image size [0, 1440]x[0, 3200]. shape=bbox order=xy limits=[1440,3200,1440,3200]
at Service.locate (/Applications/Midscene Studio Beta.app/Contents/Resources/app.asar/node_modules/@midscene/core/dist/lib/service/index.js:107:29)
at processTicksAndRejections (node:internal/process/task_queues:104:5)
at Object.executor (/Applications/Midscene Studio Beta.app/Contents/Resources/app.asar/node_modules/@midscene/core/dist/lib/agent/task-builder.js:291:36)
at TaskRunner.flush (/Applications/Midscene Studio Beta.app/Contents/Resources/app.asar/node_modules/@midscene/core/dist/lib/task-runner.js:186:35)
at TaskExecutor.runAction (/Applications/Midscene Studio Beta.app/Contents/Resources/app.asar/node_modules/@midscene/core/dist/lib/agent/tasks.js:366:17)
at runAiAct (/Applications/Midscene Studio Beta.app/Contents/Resources/app.asar/node_modules/@midscene/core/dist/lib/agent/agent.js:463:46)
at AndroidAgent.aiAct (/Applications/Midscene Studio Beta.app/Contents/Resources/app.asar/node_modules/@midscene/core/dist/lib/agent/agent.js:496:22)
at executeAction (/Applications/Midscene Studio Beta.app/Contents/Resources/app.asar/node_modules/@midscene/playground/dist/lib/common.js:210:20)
at /Applications/Midscene Studio Beta.app/Contents/Resources/app.asar/node_modules/@midscene/playground/dist/lib/server.js:2159:35

Reproduce link

no

Reproduce Steps

使用的是桌面版midscene,发现在点击边缘元素时常出现这个问题,模型用的是gpt-5.6

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions