How to use ascraper_playwright_scroll for pagination

[zymmlalala]

How to use ascraper_playwright_scroll for pagination? Here, when I configure backend=playwright_scroll, I encounter an error at the location of dynamic_import(backend, message). I would like to know how to solve this problem.

[dosubot]

Hi @zymmlalala! I'm Dosu and I’m helping the Scrapegraph-ai team.

The error happens because backend="playwright_scroll" is not a valid module name—only "playwright" or "selenium" are supported backends. To use scrolling and pagination, set backend="playwright" and then use the ascrape_playwright_scroll method, which is designed for this purpose. This method lets you control how far and how long to scroll, and whether to scroll to the bottom or for a set time. Make sure you have the playwright package installed.

Example usage:

loader = ChromiumLoader(
    urls=[your_url],
    backend="playwright",
    headless=True,
    browser_name="chromium",
)
html = await loader.ascrape_playwright_scroll(
    url=your_url,
    timeout=30,           # seconds to scroll
    scroll=15000,         # pixels per scroll
    sleep=2,              # seconds to wait after each scroll
    scroll_to_bottom=True # set to True to scroll until the page stops changing
)

You can adjust timeout, scroll, and sleep to fit your pagination needs. The scrolling logic is built into the ascrape_playwright_scroll method, so you don't need a special backend for it—just use "playwright" as the backend and call the method as shown above.
Reference: ChromiumLoader and ascrape_playwright_scroll implementation

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}