Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit concurrent context count (PLAYWRIGHT_MAX_CONTEXTS setting) #95

Merged
merged 17 commits into from
May 22, 2022

Conversation

elacuesta
Copy link
Member

@elacuesta elacuesta commented May 18, 2022

Add a setting (PLAYWRIGHT_MAX_CONTEXTS) to limit the amount of concurrent contexts allowed at any given time.

Closes #79

Tasks:

  • Implementation
  • Example
  • Tests
  • Docs (PLAYWRIGHT_MAX_CONTEXTS setting - stress the need to close contexts manually if set, otherwise the crawl can just get stuck waiting for contexts to close)

@elacuesta elacuesta added the enhancement New feature or request label May 18, 2022
@codecov
Copy link

codecov bot commented May 18, 2022

Codecov Report

Merging #95 (7cd3201) into master (c32d720) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##            master       #95   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            4         4           
  Lines          322       333   +11     
=========================================
+ Hits           322       333   +11     
Impacted Files Coverage Δ
scrapy_playwright/handler.py 100.00% <100.00%> (ø)

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

@elacuesta elacuesta marked this pull request as ready for review May 22, 2022 03:34
@elacuesta elacuesta changed the title Max contexts Limit concurrent context count May 22, 2022
@elacuesta elacuesta changed the title Limit concurrent context count Limit concurrent context count (PLAYWRIGHT_MAX_CONTEXTS setting) May 22, 2022
@elacuesta elacuesta merged commit 2e66b9e into master May 22, 2022
@elacuesta elacuesta deleted the max-contexts branch May 22, 2022 03:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

max context
1 participant