Multithreading/parallelize?

[Niemand112233]

Hi,
Your software works great, but it is a little bit slow when searching for queries on google scholar. Is it possible to parallelize for example the search on the single pages?

[ferru97]

Hi @Niemand112233,
Yes for the next version (as soon as I have time or new collaborators) I'll try to introduce multithreading(even though it's not the best thing of python) at least during the downloads so that the tool will continue searching while downloading papers.

There is also a random timeout 1-10sec after each request to Crossref (When you see on the prompt "Searching paper x of y on Crossref.."). I've put it as I think that it helps to not get blocked by crossref for API abuse. On the next version, I think I'll put this timeout optional

[creepy-pasta101]

I guess mutlithreading can be introduced by adding crossref query and scihub downloads alternatively

[ferru97]

Yes of course, even if the gain is not maximum

[ferru97]

Hi,
sorry for the late reply, but PyPaperBot had been on standby until now.
I've decided to resume development to improve it.

I've created a Telegram channel where you can suggest improvements, report bugs, or request custom data mining scripts. Feel free to join if you're interested with the following LINK

Thanks!