We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
1 parent 2d36378 commit d5f23f2Copy full SHA for d5f23f2
README.md
@@ -64,9 +64,11 @@ $ pip install -e human-eval
64
65
Here's our evaluation result on HumanEval dataset:
66
67
+Note: our model can have a comparable accuracy with Codex of similar model size.
68
+
69
|Model|Pass@1|Pass@10|Pass@100|
70
|:------:|:---:|:---:|:---:|
-|PyCodeGPT-110M |8.32% |13.53% |18.3% |
71
+|PyCodeGPT-110M |**8.32%** |**13.53%** |**18.3%** |
72
|||||
73
|GPT-Neo 125M |0.75% |1.88% |2.97% |
74
|GPT-Neo 1.3B |4.97% |7.47% |16.3% |
@@ -99,7 +101,6 @@ Here's our evaluation result on HumanEval dataset:
99
101
|PolyCoder 400M |2.96% |5.29% |11.59% |
100
102
|PolyCoder 2.7B |5.59% |9.84% |17.68% |
103
-As you can see, our PyCode 110M model is comparable with Codex 85M.
104
=======
105
106
## Reference
0 commit comments