Skip to content

Commit 356cc8a

Browse files
BitcoinNLPerBitcoinNLPer
authored
Add files via upload
1 parent fe6b50d commit 356cc8a

File tree

1 file changed

+208
-0
lines changed

1 file changed

+208
-0
lines changed

‎apicoder/README.md

+208
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,208 @@
1+
# Private Library Oriented Code Generation
2+
3+
This project is the code, data, benchmarks, and models for our paper titled [When Language Model Meets Private Library](https://arxiv.org/pdf/2206.06888.pdf).
4+
5+
---
6+
7+
## Abstract
8+
9+
With the rapid development of pre-training techniques, a number of language models have been pre-trained on large-scale code corpora and perform well in code generation. In this paper, we investigate how to equip the pre-trained language models with the ability of code generation for private libraries. In practice, it is common for programmers to write code using private libraries. However, this is a challenge for language models, since they have never seen private APIs during training. Motivated by the fact that private libraries usually come with elaborate API documentations, we propose a novel framework with two modules: the APIRetriever finds useful APIs, and then the APICoder generates code using these APIs. For APIRetriever, we present a dense retrieval system and also design a friendly interaction to involve uses. For APICoder, we can directly use the off-the-shelf language models like CODEGEN, and we also continually pre-trained the base model on a code corpus containing API information. Both modules are trained with data from public libraries, and can be generalized to private ones. Furthermore, we craft three benchmarks for private libraries, named TorchDataEval, MonkeyEval, and BeatNumEval. Experimental results demonstrate the impressive performance of our framework.
10+
11+
<img src=https://s3.bmp.ovh/imgs/2022/09/27/624d82ddc3045aea.png width=450 />
12+
13+
## Architecture of this repository
14+
15+
As this project consists of a lot of modules, we have drawn the logic diagram between data and code for you to easily understand it.
16+
17+
> You can view this diagram in line with the following project directory at a glance.
18+
19+
<img src=https://s3.bmp.ovh/imgs/2022/09/27/b81213d3e5d841b2.png width=1000 />
20+
21+
Figure1: Logic flow diagram of our code and data, where the gray background denotes the file (e.g, model, data), the red and blue letters mean the relative path of the file or code.
22+
23+
## Project Directory
24+
```shell
25+
├── APIRetriever
26+
│ ├── LICENSE
27+
│ ├── README.md
28+
│ ├── build
29+
│ ├── data
30+
│ │ ├── inference
31+
│ │ │ ├── beatnum_api.json
32+
│ │ │ ├── beatnum_api.pt
33+
│ │ │ ├── beatnum_comment.json
34+
│ │ │ ├── beatnum_comment.pt
35+
│ │ │ ├── beatnum_id_score.trec
36+
│ │ │ ├── beatnum_id_score.txt
37+
│ │ │ ├── monkey_api.json
38+
│ │ │ ├── monkey_api.pt
39+
│ │ │ ├── monkey_comment.json
40+
│ │ │ ├── monkey_comment.pt
41+
│ │ │ ├── monkey_id_score.trec
42+
│ │ │ ├── monkey_id_score.txt
43+
│ │ │ ├── numpy_api.json
44+
│ │ │ ├── numpy_api.pt
45+
│ │ │ ├── numpy_comment.json
46+
│ │ │ ├── numpy_comment.pt
47+
│ │ │ ├── numpy_id_score.trec
48+
│ │ │ ├── numpy_id_score.txt
49+
│ │ │ ├── pandas_api.json
50+
│ │ │ ├── pandas_api.pt
51+
│ │ │ ├── pandas_comment.json
52+
│ │ │ ├── pandas_comment.pt
53+
│ │ │ ├── pandas_id_score.trec
54+
│ │ │ ├── pandas_id_score.txt
55+
│ │ │ ├── torchdata_api.json
56+
│ │ │ ├── torchdata_api.pt
57+
│ │ │ ├── torchdata_comment.json
58+
│ │ │ ├── torchdata_comment.pt
59+
│ │ │ ├── torchdata_id_score.trec
60+
│ │ │ └── torchdata_id_score.txt
61+
│ │ └── train
62+
│ │ ├── processed-train-data
63+
│ │ └── unprocessed-train-data
64+
│ ├── outputs
65+
│ ├── requirements.txt
66+
│ ├── scripts
67+
│ │ ├── extract_retrieval_api_corpus.py
68+
│ │ ├── run_extract_apiretriever_corpus.sh
69+
│ │ ├── run_prepare_test_private_code.py
70+
│ │ └── run_prepare_train_private_code.py
71+
│ ├── setup.py
72+
│ └── src
73+
│ ├── dense
74+
│ ├── run_encode_2.sh
75+
│ ├── run_search_3.sh
76+
│ ├── run_train_1.sh
77+
│ └── run_trec_format_4.sh
78+
├── CodeGenAPI
79+
│ ├── APICoder
80+
│ │ ├── get_api_info_by_name.py
81+
│ │ └── get_lib_comment_for_eval.py
82+
│ ├── README.md
83+
│ ├── eval_baseline.py
84+
│ ├── eval_private.py
85+
│ ├── nl2code
86+
│ │ ├── __init__.py
87+
│ │ ├── code_dataset.py
88+
│ │ ├── code_dataset_codegen.py
89+
│ │ ├── configuration_codegen.py
90+
│ │ ├── hf_trainer.py
91+
│ │ ├── indexed_dataset.py
92+
│ │ └── modeling_codegen.py
93+
│ ├── requirements.txt
94+
│ ├── run_evaluating_codes.sh
95+
│ ├── run_generating_codes.sh
96+
│ ├── run_private.py
97+
│ ├── run_private.sh
98+
│ └── scripts
99+
│ ├── __init__.py
100+
│ ├── encode_private_data.py
101+
│ ├── extract_api.py
102+
│ ├── file_utils.py
103+
│ ├── get_comments_from_evallibs.py
104+
│ ├── get_libs_info_from_code.py
105+
│ ├── make_human_in_the_loop_test_corpus.py
106+
│ ├── multiprocessing_utils.py
107+
│ ├── pycode_visitor.py
108+
│ ├── requirements.txt
109+
│ ├── run_details_apis.sh
110+
│ ├── run_encode_private_data.sh
111+
│ ├── run_extract_apis.sh
112+
│ └── run_extract_details_from_apis.py
113+
├── README.md
114+
├── data
115+
│ ├── API-Doc
116+
│ │ └── README.md
117+
│ ├── Cleaned-Private-Code-Files
118+
│ │ └── README.md
119+
│ ├── CodeGenAPI
120+
│ │ └── README.md
121+
│ └── EncodedCorpus4CodeGenAPI
122+
│ └── README.md
123+
└── private-eval
124+
├── LICENSE
125+
├── README.md
126+
├── data
127+
│ ├── TorchData_no.API_number_0.CodeGen.hm_False.machine.t0.1.p0.9.l100.n1.samples.jsonl
128+
│ ├── XXXAPIEval-make sense.ipynb
129+
│ ├── numpy_keyword_mapping.json
130+
│ ├── numpy_keywords.jsonl
131+
│ ├── pandas_keyword_mapping.json
132+
│ ├── pandas_keywords.jsonl
133+
│ ├── real_beatnum_eval_v3.jsonl.gz
134+
│ ├── real_beatnum_eval_v3_api_1.jsonl.gz
135+
│ ├── real_beatnum_eval_v3_api_2.jsonl.gz
136+
│ ├── real_beatnum_eval_v3_api_3.jsonl.gz
137+
│ ├── real_beatnum_eval_v3_api_5.jsonl.gz
138+
│ ├── real_beatnum_eval_v3_api_n.jsonl.gz
139+
│ ├── real_beatnum_eval_v3_human_labelled.jsonl.gz
140+
│ ├── real_monkey_eval_v3.jsonl.gz
141+
│ ├── real_monkey_eval_v3_api_1.jsonl.gz
142+
│ ├── real_monkey_eval_v3_api_2.jsonl.gz
143+
│ ├── real_monkey_eval_v3_api_3.jsonl.gz
144+
│ ├── real_monkey_eval_v3_api_5.jsonl.gz
145+
│ ├── real_monkey_eval_v3_api_n.jsonl.gz
146+
│ ├── real_monkey_eval_v3_human_labelled.jsonl.gz
147+
│ ├── real_numpy_eval_v3.jsonl.gz
148+
│ ├── real_numpy_eval_v3_api_1.jsonl.gz
149+
│ ├── real_numpy_eval_v3_api_2.jsonl.gz
150+
│ ├── real_numpy_eval_v3_api_3.jsonl.gz
151+
│ ├── real_numpy_eval_v3_api_5.jsonl.gz
152+
│ ├── real_numpy_eval_v3_api_n.jsonl.gz
153+
│ ├── real_pandas_eval_v3.jsonl.gz
154+
│ ├── real_pandas_eval_v3_api_1.jsonl.gz
155+
│ ├── real_pandas_eval_v3_api_2.jsonl.gz
156+
│ ├── real_pandas_eval_v3_api_3.jsonl.gz
157+
│ ├── real_pandas_eval_v3_api_5.jsonl.gz
158+
│ ├── real_pandas_eval_v3_api_n.jsonl.gz
159+
│ ├── real_torchdata_eval_v3.jsonl.gz
160+
│ ├── real_torchdata_eval_v3_api_1.jsonl.gz
161+
│ ├── real_torchdata_eval_v3_api_1_make_sense.jsonl.gz
162+
│ ├── real_torchdata_eval_v3_api_2.jsonl.gz
163+
│ ├── real_torchdata_eval_v3_api_2_make_sense.jsonl.gz
164+
│ ├── real_torchdata_eval_v3_api_3.jsonl.gz
165+
│ ├── real_torchdata_eval_v3_api_3_make_sense.jsonl.gz
166+
│ ├── real_torchdata_eval_v3_api_5.jsonl.gz
167+
│ ├── real_torchdata_eval_v3_api_5_make_sense.jsonl.gz
168+
│ ├── real_torchdata_eval_v3_api_n.jsonl.gz
169+
│ ├── real_torchdata_eval_v3_api_n_make_sense.jsonl.gz
170+
│ ├── real_torchdata_eval_v3_human_labelled.jsonl.gz
171+
│ └── real_torchdata_eval_v3_human_labelled_make_sense.jsonl.gz
172+
├── private_eval
173+
│ ├── __init__.py
174+
│ ├── data.py
175+
│ ├── evaluate_functional_correctness.py
176+
│ ├── evaluation.py
177+
│ └── execution.py
178+
├── requirements.txt
179+
└── setup.py
180+
```
181+
182+
## Download
183+
Download the API-Doc file below, unzip it and put it under path `data/API-Doc`.
184+
185+
> [Click here to download API-Doc](https://drive.google.com/file/d/1hJak_upTdrSHsI0DXeeH_aMBzKXn87AA/view?usp=sharing).
186+
```
187+
cd data/API-Doc
188+
unzip API-Doc.zip
189+
```
190+
191+
Also, you should download the CodeGenAPI file below, unzip it and put it under path `data/CodeGenAPI`.
192+
193+
> [Click here to download CodeGenAPI-350M-mono](https://drive.google.com/file/d/1Az02eqYM4wTXz758_RxEqnnK5Bv3BUVv/view?usp=sharing)
194+
```
195+
cd data/CodeGenAPI
196+
unzip CodeGenAPI-350M-mono.zip
197+
```
198+
199+
## Citation
200+
If you find our work useful, please cite the paper:
201+
```
202+
@inproceedings{APICoder,
203+
title={When Languange Model Meets Private Library},
204+
author={Zan, Daoguang and Chen, Bei and Lin, Zeqi and Guan, Bei and Wang, Yongji and Lou, Jian-Guang},
205+
booktitle={ArXiv},
206+
year={2022}
207+
}
208+
```

0 commit comments

Comments
 (0)