GitHub - psreid/ArXiv_Miner: ArXiv Miner Designed to scrape the pdf file from export

psreid / ArXiv_Miner Public

Notifications You must be signed in to change notification settings
Fork 2
Star 1

ArXiv Miner Designed to scrape the pdf file from export_ArXiv sources.

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 37 Commits
.idea		.idea
ArXiv.py		ArXiv.py
BDT_query_workflow.py		BDT_query_workflow.py
Mine_Tool.py		Mine_Tool.py
Miner_base.py		Miner_base.py
README.txt		README.txt
hepex_ArXiv.csv		hepex_ArXiv.csv
requirements.txt		requirements.txt
setup.py		setup.py

Repository files navigation

# ArXiv Miner is desinged to: Retrive, Save, and Query ArXiv PDFs
# Inlcuded in this packege is a csv file of all Hep-ex ID's
# However all of the ID's a re regularly maintained on Kaggle
# https://www.kaggle.com/Cornell-University/arxiv
# ------------

# Setup instructions for CEDAR
# ------------
# setup a virtual env with the following commands

# module load python/3.7
# virtualenv --no-download ARXENV
# source ARXENV/bin/activate

# Navigate to ArXiv_Miner source

# pip3 install numpy
# pip3 install pandas
# pip3 install pdftotext
# python3 setup.py install (not necessary command on cedar)
#


# Then you can run BDT_workflow script from job submissions