Skip to content

Commit d5f23f2

Browse files
authored
Update README.md
1 parent 2d36378 commit d5f23f2

File tree

1 file changed

+3
-2
lines changed

1 file changed

+3
-2
lines changed

README.md

+3-2
Original file line numberDiff line numberDiff line change
@@ -64,9 +64,11 @@ $ pip install -e human-eval
6464

6565
Here's our evaluation result on HumanEval dataset:
6666
67+
Note: our model can have a comparable accuracy with Codex of similar model size.
68+
6769
|Model|Pass@1|Pass@10|Pass@100|
6870
|:------:|:---:|:---:|:---:|
69-
|PyCodeGPT-110M |8.32% |13.53% |18.3% |
71+
|PyCodeGPT-110M |**8.32%** |**13.53%** |**18.3%** |
7072
|||||
7173
|GPT-Neo 125M |0.75% |1.88% |2.97% |
7274
|GPT-Neo 1.3B |4.97% |7.47% |16.3% |
@@ -99,7 +101,6 @@ Here's our evaluation result on HumanEval dataset:
99101
|PolyCoder 400M |2.96% |5.29% |11.59% |
100102
|PolyCoder 2.7B |5.59% |9.84% |17.68% |
101103
102-
As you can see, our PyCode 110M model is comparable with Codex 85M.
103104
=======
104105
105106
## Reference

0 commit comments

Comments
 (0)