Skip to content

Commit fb94429

Browse files
authored
Update GLM.md
1 parent 25d116f commit fb94429

File tree

1 file changed

+12
-3
lines changed

1 file changed

+12
-3
lines changed

docs/GLM.md

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -17,10 +17,19 @@ The key features of GLM include:
1717
- Feed-forward network is replaced with a linear layer
1818

1919
## Performance of GLM
20-
The performance of GLM on SuperGLUE benchmark is listed below:
20+
### SuperGLUE
21+
22+
1. With multi-task pretraining, GLM-Doc and GLM-Sent perform slightly worse than GLM-Large, but still outperform BERT-Large and UniLM-Large.
23+
2. Among multitask models, GLM-Sent outperforms GLM-Doc by 1.1% on average. Increasing GLM-Doc’s parameters to 410M (1.25×BERT-Large) leads to better performance than GLM-Large. GLM with 515M parameters (1.5×BERT-Large) can perform even better.
24+
2125
<div align=center><img src="img/glm_results2.png"></div>
2226

23-
As shown in the chart below, GLM-XXlarge model has achieved an avarage score of 79.297 on the CLUE1.0 benchmark, and outperforms the rest of the models on TNEWs, OCNLI, CLUEWSC, and C3.
27+
### CLUE
28+
29+
1. GLM-XXLarge has has achieved an avarage score of 79.297 which is improved significantly in multiple tasks. In the selected 3 general +2 business evaluation tasks, the average improvement is 2.47pp.
30+
2. Among the tasks in the CLUE1.0, except for the CMRC task, the average improvement is 1.56pp, of which the C3 and OCNLI data sets have improved significantly (+9.9PP, +2.84PP).
31+
32+
2433
<div align=center><img src="img/glm_performance.png"></div>
2534

2635
## Supported pre-trained GLM models
@@ -39,4 +48,4 @@ see [Tutorial 5: Supported models](https://model.baai.ac.cn/models)
3948
<div align=center><img src="img/glm_example_4.png" width="400px"></div>
4049

4150
4) The self-attention mask that realizes both autoencoding upon corrupted text and autoregressive upon the masked spans
42-
<div align=center><img src="img/glm_example_5.png" width="400px"></div>
51+
<div align=center><img src="img/glm_example_5.png" width="400px"></div>

0 commit comments

Comments
 (0)