Skip to content

fix: Make context helpers work in FailedRequestHandler and ErrorHandler #1570

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
Pijukatel merged 8 commits into from
Nov 19, 2025
Merged

fix: Make context helpers work in FailedRequestHandler and ErrorHandler #1570

Pijukatel merged 8 commits into from
Nov 19, 2025

Conversation

@Pijukatel
Copy link
Collaborator

@Pijukatel Pijukatel commented Nov 18, 2025 •
edited
Loading

Description

  • Make context helpers push_data,get_key_value_store, and add_requests work in FailedRequestHandler and ErrorHandler. Previously, they had no effect as the result of a failed request was never committed.

Issues

Testing

  • Added unit tests

Checklist

  • CI passed

@github-actions github-actions bot added this to the 128th sprint - Tooling team milestone Nov 18, 2025
@github-actions github-actions bot added t-tooling Issues with this label are in the ownership of the tooling team. tested Temporary label used only programatically for some analytics. labels Nov 18, 2025
@Pijukatel Pijukatel marked this pull request as ready for review November 18, 2025 15:17
janbuchar
janbuchar reviewed Nov 18, 2025
View reviewed changes
@Pijukatel Pijukatel requested a review from janbuchar November 19, 2025 10:42
janbuchar
janbuchar reviewed Nov 19, 2025
View reviewed changes
Copy link
Collaborator

@janbuchar janbuchar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's starting to grow on me 🤷 What do you think @Pijukatel?

Pijukatel and others added 2 commits November 19, 2025 12:58
@Pijukatel @janbuchar
Update src/crawlee/crawlers/_basic/_basic_crawler.py
b29f34d 
Co-authored-by: Jan Buchar <jan.buchar@apify.com>
@Pijukatel
Review comments
7dc6c7e
@Pijukatel Pijukatel requested a review from janbuchar November 19, 2025 12:02
janbuchar
janbuchar approved these changes Nov 19, 2025
View reviewed changes
Copy link
Collaborator

@janbuchar janbuchar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Mantisus
Mantisus approved these changes Nov 19, 2025
View reviewed changes
Copy link
Collaborator

@Mantisus Mantisus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Pijukatel Pijukatel changed the title fix: Make context helpers work in FailedRequestHandler fix: Make context helpers work in FailedRequestHandler and ErrorHandler Nov 19, 2025
@Pijukatel Pijukatel merged commit b830019 into master Nov 19, 2025
45 of 46 checks passed
@Pijukatel Pijukatel deleted the failed-request-handler-push-data branch November 19, 2025 16:12
@mabreuortega mabreuortega mentioned this pull request Dec 16, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@janbuchar janbuchar janbuchar approved these changes

+1 more reviewer

@Mantisus Mantisus Mantisus approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@Pijukatel Pijukatel

Labels

t-tooling Issues with this label are in the ownership of the tooling team. tested Temporary label used only programatically for some analytics.

Projects

None yet

Milestone

128th sprint - Tooling team

Development

Successfully merging this pull request may close these issues.

failed_request_handler runs and logs but context.push_data(...) does not write to dataset (PlaywrightCrawler)

4 participants

@Pijukatel @janbuchar @Mantisus