chromesession
is a Python package that provides a convenient contextmanager for managing selenium
chrome sessions.
In addition, a CachedSession
is provided to directly cache the driver responses.
pip install chromesession
To use the chromesession.chrome
context manager with selenium
, the chromedriver must be installed on the system.
Alternatively, you can install the latest chromedriver
as an extra.
pip install chromesession[driver]
Cache the specified URLs by fetching them via Selenium and saving the responses.
from pathlib import Path
from chromesession import CachedSession, chrome
def caching(*urls: str) -> Path:
"""
Cache the specified URLs by fetching them via Selenium and saving the responses.
"""
cachfile = "caching.sqlite"
with CachedSession(cache_name=cachfile) as session:
with chrome(verbose=False) as driver:
for url in urls:
if url in session:
print(f"{url=} already cached.")
continue
try:
driver.get(url)
session.save_driver(driver)
except Exception as e:
print(f"{url=} failed to cache: {e}", exc_info=True)
else:
print(f"{url=} saved in cache.")
return Path(cachfile)
if __name__ == "__main__":
caching("https://example.com/", "https://example.com/")
Install optional dependencies using extras.
extra | installation | dependency |
---|---|---|
all | pip install chromesession[all] |
Install all extras. |
driver | pip install chromesession[driver] |
|
bs4 | pip install chromesession[bs4] |