Skip to content

feat: Add RQ id, name, alias args to add_requests and enqueue_links methods#1413

Merged
vdusek merged 7 commits into
apify:masterfrom
Mantisus:up-add-requests
Sep 22, 2025
Merged

feat: Add RQ id, name, alias args to add_requests and enqueue_links methods#1413
vdusek merged 7 commits into
apify:masterfrom
Mantisus:up-add-requests

Conversation

@Mantisus

Copy link
Copy Markdown
Collaborator

Description

  • Add support for specifying RQ dst for add_requests and enqueue_links methods

Issues

@Mantisus Mantisus self-assigned this Sep 16, 2025

@vdusek vdusek left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also include rq_id?

My thinking was that this should apply only to the context-level helpers, not the crawler-level ones. At the crawler level, they should always use the crawler's configured request manager.

@Mantisus

Copy link
Copy Markdown
Collaborator Author

Should we also include rq_id?

Yes, I think we should, for consistency of supported parameters. Added rq_id

My thinking was that this should apply only to the context-level helpers, not the crawler-level ones.

Yes, this PR adds support for new parameters, specifically in helpers.

@Mantisus Mantisus requested a review from vdusek September 20, 2025 16:46

@vdusek vdusek left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few details, my comments apply to their other occurrences as well.

Comment thread src/crawlee/_types.py Outdated
Comment thread src/crawlee/_types.py Outdated
Comment thread src/crawlee/_types.py Outdated
Comment thread src/crawlee/crawlers/_basic/_basic_crawler.py Outdated
Comment thread tests/unit/crawlers/_basic/test_basic_crawler.py
Comment thread tests/unit/crawlers/_basic/test_basic_crawler.py
Comment thread tests/unit/crawlers/_basic/test_basic_crawler.py
Comment thread tests/unit/crawlers/_beautifulsoup/test_beautifulsoup_crawler.py
Mantisus and others added 4 commits September 21, 2025 19:43
Co-authored-by: Vlada Dusek <v.dusek96@gmail.com>
Co-authored-by: Vlada Dusek <v.dusek96@gmail.com>
@Mantisus Mantisus requested a review from vdusek September 21, 2025 20:39

@vdusek vdusek left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vdusek vdusek changed the title feat: Add rq_name and rq_alias params for add_requests and enqueue_links methods feat: Add RQ id, name, alias args to add_requests and enqueue_links methods Sep 22, 2025
@vdusek vdusek merged commit 1cae2bc into apify:master Sep 22, 2025
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for specifying RQ dst for add_requests and enqueue_links methods

3 participants