Feature Request | 功能需求

[tuchief]

1、Timed task
2、Online packaging and deployment
3、Log monitoring alarm

[my8100]

@tuchief

2、Online packaging and deployment

Have you tried the Projects > Deploy page?

[tuchief]

I know what you said, but this method should be manually packaged into egg, and then uploaded, and can not directly package the source project into egg, automatically upload it?

[my8100]

I know what you said, but this method should be manually packaged into egg, and then uploaded, and can not directly package the source project into egg, automatically upload it?

Ok, I would try to figure out a better way to eggify local projects.

[tuchief]

You can refer to https://github.com/Gerapy/Gerapy

[my8100]

v0.9.9: Add auto eggifying

[tuchief]

Wow, the response is fast! Expecting the same timing tasks

[wymen2018]

1. You can select multiple crawlers when you want to set a timed task. You can select multiple crawlers to start at each time period after selecting the time point.
   Imagine that I have 100 crawlers in a project. This feature does provide convenience and management.
   2, as above, because there are a hundred crawlers, so can you have the function of custom label, that is, I can use the label screen and view the running status of the specified category of crawlers, and timed tasks
   Thanks!

[my8100]

You can select multiple crawlers when you want to set a timed task. You can select multiple crawlers to start at each time period after selecting the time point.
Imagine that I have 100 crawlers in a project. This feature does provide convenience and management.

You mean there are 100 spiders in a project and you want to schedule some of them to run periodically?

as above, because there are a hundred crawlers, so can you have the function of custom label, that is, I can use the label screen and view the running status of the specified category of crawlers, and timed tasks

What about labeling some related jobs with a specific jobid?

[wymen2018]

You mean there are 100 spiders in a project and you want to schedule some of them to run periodically?

No, Simultaneously select multiple spiders to set one timed task.

What about labeling some related jobs with a specific jobid?

Labels will be better because they can be more visualized according to their own classification

[my8100]

OK, I would take it into account when implementing this feature. Thanks for your advice!

[my8100]

v1.0.0rc1: Add Email Notice, with multi-triggers provided, including:

• ON_JOB_RUNNING_INTERVAL

• ON_JOB_FINISHED

• When reaching the threshold of specific kind of log: ['CRITICAL', 'ERROR', 'WARNING', 'REDIRECT', 'RETRY', 'IGNORE'], in the meanwhile, you can ask ScrapydWeb to stop/forcestop current job automatically.

Get it via the pip install scrapydweb==1.0.0rc1 command.

Email sample: