Enabled abstract retrieval by different types of Scopus ID - #71
Conversation
|
Hi @mbrcic, this is very cool, thanks! I did not know this was possible. |
|
Hi @Michael-E-Rose , ID type auto-discovery is a good idea! DOI and EID are easily distinguishable, as well as PII. I've just added the auto-discovery into the code and it works great for my IDs of all types. It can be tweaked in future if there are better discriminators between Scopus ID and PubMed ID. |
|
Just to add, I did not remove the ID_type parameter because I like to give users more flexibility. If ID_type is set to 'auto' (which is the default value), then the type is inferred from the ID. User can also manually specify the type of ID through the same parameter. |
|
Yes, that's probably a smart move. But rather than |
|
I've changed ID_type default value to None, it does make more sense. |
| from scopus.utils.get_content import * | ||
| from scopus.utils.get_encoded_text import * | ||
| from scopus.utils.startup import * | ||
| from scopus.utils.discover_id_type import * No newline at end of file |
There was a problem hiding this comment.
There needs to be a blank line after this
| raise ValueError('ID_type parameter must be one of ' + | ||
| ', '.join(allowed_id_types)) | ||
|
|
||
| qfile = join(config.get('Directories', 'AbstractRetrieval'), EID.replace('/','_')) |
There was a problem hiding this comment.
why the .replace('/','_')?
There was a problem hiding this comment.
Still open question
There was a problem hiding this comment.
This is due to caching in function scopus.utils.get_content, line 85 which tries to create the file.
In Unix-based systems '/' is used as a path separator so it can't be used in file names. This is only a problem in the case of DOI, which always contains '/'.
There was a problem hiding this comment.
Though, I admit that replacement character '_' is kind-of arbitrarily chosen. And EID could be defensively casted to string when calling for replacement.
There was a problem hiding this comment.
Ah, I see. That's why in the search classes we cache files with the hashed-version of the filename. I think the underscore is okay. Could you also update the note in the docstring, please?
|
|
||
| ID_type: str (optional, default=auto) | ||
| The overload type of used ID. On Scopus it can be one of | ||
| {'eid','pii','scopus_id','pubmed_id','doi'}. If using option |
There was a problem hiding this comment.
No set nottation, please.
| from scopus import config | ||
| from scopus.utils import get_content | ||
|
|
||
| from scopus.utils import discover_id_type |
There was a problem hiding this comment.
can be merged with above line: from scopus.utils import discover_id_type, get_content
There was a problem hiding this comment.
This one still needs to be adressed
| cases with only 16 for old converted articles. | ||
|
|
||
| """ | ||
| if ID.startswith('2-s2.0-'): |
There was a problem hiding this comment.
This will run into AttributeErrors when ID is already numeric. You should write str(ID).startswith('2-s2.0-')
There was a problem hiding this comment.
All the conditional statements in this function lean on the assumption that ID is string. It is probably best to defensively cast it to string at the beginning of the function.
There was a problem hiding this comment.
Yes, or even before that, in __init__. Because in the case of pubmed or scopus ID, users can supply an integer rather than a string.
There was a problem hiding this comment.
While it is true that both pubmed and scopus IDs contain only digits, I have come across scopus IDs with leading zeros. If interpreting or supplying them as integers with leading zeros, it will change the token interpretation and that renders the ID invalid. Namely, leading zeros in python 2 force interpretation as an octal number, if possible. Python 3 does not allow leading zeros in numbers and raises SyntaxError. The scopus ID supplied as a number with manually omitted leading zeros is an invalid resource identifier.
So I would say that scopus ID is a string consisting only of digits. You can try it out on the following 10-digit scopus ID:
https://api.elsevier.com/content/abstract/scopus_id/0028981867
Pubmed ID is simply a number which can be supplied as such.
I have put this in the notes of detect_id_type function. And I have casted IDs to strings in both functions.
|
Thanks, it's getting there! There are few open things however. When you have addressed them, I merge the PR. |
1fbc71a to
8bb8035
Compare
…es, and casted ID to str.
8bb8035 to
2d0081e
Compare
|
Thanks a lot for this work, @mbrcic! I just merged the PR. Somewhen next week I will make this live with scopus 1.2. There's one other minor thing in another class I want to address. |
2 participants
I have generalized abstract retrieval by other types of ID, in addition to already supported EID. Abstracts can now also be retrieved by DOI, PII, PubMed ID, and Scopus ID. This generalization was necessary for my project.