[Bug]: Setting MAGIC parameters causes [ERROR]... × Error updating image dimensions: Page.evaluate: Execution context was destroyed, most likely because of a navigation

[xingyongma]

crawl4ai version

0.4.247

Expected Behavior

Jump to the target webpage

Current Behavior

In the screenshot, "accept cookie" is visible, but the webpage with the target content is not seen.

Is this reproducible?

Yes

Inputs Causing the Bug

Steps to Reproduce

set "magic = True" can reproduce the issue.

Code snippets

import asyncio
import base64
import os
from crawl4ai import AsyncWebCrawler, CacheMode


async def main():
    async with AsyncWebCrawler(
            headless=True,  # Set to False to see what is happening
            verbose=True,
            # New user agent mode that allows you to specify 
            # the device type and os type, and get a random user agent
            user_agent_mode="random",
            user_agent_generator_config={
                "device_type": "mobile",
                "os_type": "android"
            },
    ) as crawler:
        result = await crawler.arun(
            url='https://link.springer.com/journal/12188',
            cache_mode=CacheMode.BYPASS,
            html2text = {
                "ignore_links": True
            },
            page_timeout=120000,
            delay_before_return_html= 2,
            magic = True,
            screenshot=True
        )
        
        if result.success:
            print(len(result.markdown_v2.raw_markdown))
            with open("./google.md", "w") as f:
                f.write(result.markdown_v2.raw_markdown)
            # Save screenshot
            with open("./screenshot.png", "wb") as f:
                f.write(base64.b64decode(result.screenshot))
                
            # Show screenshot
            #os.system(f"open {__location__}/output/screenshot.png")
            

if __name__ == "__main__":
    asyncio.run(main())

OS

macOS

Python version

3.12.6

Browser

Chrome

Browser version

No response

Error logs & Screenshots (if applicable)

To bypass antibot measures, I set magic=True when crawling all websites, but some sites throw the following error when this parameter is enabled. When I remove this parameter, everything works fine.
In addition to the magic parameter, I sometimes encounter the following error (no example provided) when using remove_overlay_elements=True on certain pages.
Do you have any suggestions?

[ERROR]... × Error updating image dimensions: Page.evaluate: Execution context was destroyed, most likely because of a navigation.