本项目从arxiv原始文件中收集两种语料:
- .tex类型的代码语料
- 多模态语料
pip install chardet docker
安装 https://github.com/arxiv-vanity/engrafo 中的docker file
准备好arxiv原始文件
python extract.py
得到的tex.jsonl是代码语料。
-
Notifications
You must be signed in to change notification settings - Fork 1
YuqiHUO/arxiv_extractor_mnbvc
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
No description, website, or topics provided.
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published