Skip to content

Commit e3d39ba

Browse files
JingfengYangJingfengYang
authored
Merge pull request Mooler0410#19 from xinyadu/main
add model and data licensing summary
2 parents 64a6f7d + 1b42e7c commit e3d39ba

File tree

1 file changed

+288
-2
lines changed

1 file changed

+288
-2
lines changed

‎README.md

Lines changed: 288 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,10 @@
55
<img src="https://camo.githubusercontent.com/64f8905651212a80869afbecbf0a9c52a5d1e70beab750dea40a994fa9a9f3c6/68747470733a2f2f617765736f6d652e72652f62616467652e737667" alt="Awesome" data-canonical-src="https://awesome.re/badge.svg" style="max-width: 100%;">
66
</p>
77

8-
A curated (still actively updated) list of practical guide resources of LLMs. It's based on our survey paper: [Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond](https://arxiv.org/abs/2304.13712). The survey is partially based on the second half of this [Blog](https://jingfengyang.github.io/gpt). We also build an evolutionary tree of modern Large Language Models (LLMs) to trace the development of language models in recent years and highlights some of the most well-known models.
8+
A curated (still actively updated) list of practical guide resources of LLMs. It's based on our survey paper: [Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond](https://arxiv.org/abs/2304.13712) and efforsts from @[xinyadu](https://github.com/xinyadu). The survey is partially based on the second half of this [Blog](https://jingfengyang.github.io/gpt). We also build an evolutionary tree of modern Large Language Models (LLMs) to trace the development of language models in recent years and highlights some of the most well-known models.
99

10-
These sources aim to help practitioners navigate the vast landscape of large language models (LLMs) and their applications in natural language processing (NLP) applications. If you find any resources in our repository helpful, please feel free to use them (don't forget to cite our paper! 😃). We welcome pull requests to refine this figure!
10+
These sources aim to help practitioners navigate the vast landscape of large language models (LLMs) and their applications in natural language processing (NLP) applications. We also include their usage restrictions based on the model and data licensing information.
11+
If you find any resources in our repository helpful, please feel free to use them (don't forget to cite our paper! 😃). We welcome pull requests to refine this figure!
1112

1213
<p align="center">
1314
<img width="600" src="./imgs/models-colorgrey.jpg"/>
@@ -26,6 +27,7 @@ These sources aim to help practitioners navigate the vast landscape of large lan
2627
```
2728

2829
## Latest News💥
30+
- We added usage and restrictions section.
2931
- We used PowerPoint to plot the figure and released the source file [pptx](./source/figure_gif.pptx) for our GIF figure. [4/27/2023]
3032
- We released the source file for the still version [pptx](./source/figure_still.pptx), and replaced the figure in this repo with the still version. [4/29/2023]
3133
- Add AlexaTM, UniLM, UniLMv2 to the figure, and correct the logo for Tk. [4/29/2023]
@@ -64,6 +66,7 @@ These sources aim to help practitioners navigate the vast landscape of large lan
6466
* [Truthfulness Alignment (Honest)](#truthfulness-alignment-honest)
6567
* [Practical Guides for Prompting (Helpful)](#practical-guides-for-prompting-helpful)
6668
* [Alignment Efforts of Open-source Communtity](#alignment-efforts-of-open-source-communtity)
69+
* [Usage and Restractions (Models and Data)](#Usage-and-Restrictions)
6770

6871
## Practical Guide for Models
6972

@@ -273,6 +276,289 @@ We build a decision flow for choosing LLMs or fine-tuned models~\protect\footnot
273276
- **MOSS**. [Repo](https://github.com/OpenLMLab/MOSS)
274277
- **Lamini**. [Repo](https://github.com/lamini-ai/lamini/)/[Blog](https://lamini.ai/blog/introducing-lamini)
275278

279+
## Usage and Restrictions
280+
281+
<!-- We build a decision flow for choosing LLMs or fine-tuned models~\protect\footnotemark for user's NLP applications. -->
282+
<!-- The decision flow helps users assess whether their downstream NLP applications at hand meet specific conditions and, based on that evaluation, determine whether LLMs or fine-tuned models are the most suitable choice for their applications. -->
283+
284+
We build a table summarizing the LLMs usage restrictions (e.g. for commercial and research purposes). In particular, we provide the information from the models and their pretraining data's perspective.
285+
We urge the users in the community to refer to the licensing information for public models and data and use them in a responsible manner.
286+
We urge the developers to pay special attention to licensing, make them transparent and comprehensive, to prevent any unwanted and unforeseen usage.
287+
288+
<table class="table table-bordered table-hover table-condensed">
289+
<thead><tr><th title="Field #1">LLMs</th>
290+
<th title="Field #2" colspan="3" align="center">Model</th>
291+
<!-- <th title="Field #3"></th> -->
292+
<!-- <th title="Field #4"></th> -->
293+
<th title="Field #5" colspan="2" align="center">Data</th>
294+
<!-- <th title="Field #6"></th> -->
295+
</tr></thead>
296+
<tbody><tr>
297+
<td> </td>
298+
<td><b>License<b></td>
299+
<td><b>Commercial Use<b></td>
300+
<td><b>Other noteable restrictions<b></td>
301+
<td><b>License<b></td>
302+
<td><b>Corpus<b></td>
303+
</tr>
304+
<tr>
305+
<td colspan="6" align="left"><b>Encoder-only</b></td>
306+
<tr>
307+
<tr>
308+
<td>BERT series of models (general domain)</td>
309+
<td>Apache 2.0</td>
310+
<td>✅</td>
311+
<td> </td>
312+
<td>Public</td>
313+
<td>BooksCorpus, English Wikipedia</td>
314+
</tr>
315+
<tr>
316+
<td>RoBERTa</td>
317+
<td>MIT license</td>
318+
<td>✅</td>
319+
<td> </td>
320+
<td>Public</td>
321+
<td>BookCorpus, CC-News, OpenWebText, STORIES</td>
322+
</tr>
323+
<tr>
324+
<td>ERNIE</td>
325+
<td>Apache 2.0</td>
326+
<td>✅</td>
327+
<td> </td>
328+
<td>Public</td>
329+
<td>English Wikipedia</td>
330+
</tr>
331+
<tr>
332+
<td>SciBERT</td>
333+
<td>Apache 2.0</td>
334+
<td>✅</td>
335+
<td> </td>
336+
<td>Public</td>
337+
<td>BERT corpus, <a href="https://aclanthology.org/N18-3011.pdf">1.14M papers from Semantic Scholar</a></td>
338+
</tr>
339+
<tr>
340+
<td>LegalBERT</td>
341+
<td>CC BY-SA 4.0</td>
342+
<td>❌</td>
343+
<td> </td>
344+
<td>Public (except data from the <a href="https://case.law/">Case Law Access Project</a>)</td>
345+
<td>EU legislation, US court cases, etc.</td>
346+
</tr>
347+
<tr>
348+
<td>BioBERT</td>
349+
<td>Apache 2.0</td>
350+
<td>✅</td>
351+
<td> </td>
352+
<td><a href="https://www.nlm.nih.gov/databases/download/terms_and_conditions.html">PubMed</a></td>
353+
<td>PubMed, PMC</td>
354+
</tr>
355+
<tr>
356+
<td colspan="6" align="left"><b>Encoder-Decoder</b></td>
357+
<tr>
358+
<tr>
359+
<td>T5</td>
360+
<td>Apache 2.0</td>
361+
<td>✅</td>
362+
<td> </td>
363+
<td>Public</td>
364+
<td>C4</td>
365+
</tr>
366+
<tr>
367+
<td>Flan-T5</td>
368+
<td>Apache 2.0</td>
369+
<td>✅</td>
370+
<td> </td>
371+
<td>Public</td>
372+
<td>C4, Mixture of tasks (Fig 2 in paper)</td>
373+
</tr>
374+
<tr>
375+
<td>BART</td>
376+
<td>Apache 2.0</td>
377+
<td>✅</td>
378+
<td> </td>
379+
<td>Public</td>
380+
<td>RoBERTa corpus </td>
381+
</tr>
382+
<tr>
383+
<td>GLM</td>
384+
<td>Apache 2.0</td>
385+
<td>✅</td>
386+
<td> </td>
387+
<td>Public</td>
388+
<td>BooksCorpus and English Wikipedia</td>
389+
</tr>
390+
<tr>
391+
<td>ChatGLM</td>
392+
<td><a href="https://github.com/THUDM/ChatGLM-6B/blob/main/MODEL_LICENSE">ChatGLM License</a></td>
393+
<td>❌</td>
394+
<td>No use for illegal purposes or military research, no harm the public interest of society</td>
395+
<td>N/A</td>
396+
<td>1T tokens of Chinese and English corpus</td>
397+
</tr>
398+
<tr>
399+
<td colspan="6" align="left"><b>Decoder-only</b></td>
400+
<tr>
401+
<td>GPT2 </td>
402+
<td><a href="https://github.com/openai/gpt-2/blob/master/LICENSE">Modified MIT License</a></td>
403+
<td>✅</td>
404+
<td>Use GPT-2 responsibly and clearly indicate your content was created using GPT-2.</td>
405+
<td>Public</td>
406+
<td>WebText</td>
407+
</tr>
408+
<tr>
409+
<td>GPT-Neo</td>
410+
<td>MIT license</td>
411+
<td>✅</td>
412+
<td> </td>
413+
<td>Public</td>
414+
<td><a href="https://pile.eleuther.ai/">Pile</a></td>
415+
</tr>
416+
<tr>
417+
<td>GPT-J</td>
418+
<td>Apache 2.0</td>
419+
<td>✅</td>
420+
<td> </td>
421+
<td>Public</td>
422+
<td>Pile</td>
423+
</tr>
424+
<tr>
425+
<td>---&gt; Dolly</td>
426+
<td>CC BY NC 4.0</td>
427+
<td>❌</td>
428+
<td> </td>
429+
<td>CC BY NC 4.0, Subject to terms of Use of the data generated by OpenAI</td>
430+
<td>Pile, Self-Instruct</td>
431+
</tr>
432+
<tr>
433+
<td>---&gt; GPT4ALL-J</td>
434+
<td>Apache 2.0</td>
435+
<td>✅</td>
436+
<td> </td>
437+
<td>Public</td>
438+
<td><a href="https://huggingface.co/datasets/nomic-ai/gpt4all-j-prompt-generations">GPT4All-J dataset</a></td>
439+
</tr>
440+
<tr>
441+
<td>Pythia</td>
442+
<td>Apache 2.0</td>
443+
<td>✅</td>
444+
<td> </td>
445+
<td>Public</td>
446+
<td>Pile</td>
447+
</tr>
448+
<tr>
449+
<td>---&gt; Dolly v2</td>
450+
<td>MIT license</td>
451+
<td>✅</td>
452+
<td> </td>
453+
<td>Public</td>
454+
<td>Pile, databricks-dolly-15k</td>
455+
</tr>
456+
<tr>
457+
<td>OPT</td>
458+
<td><a href="https://github.com/facebookresearch/metaseq/blob/main/projects/OPT/MODEL_LICENSE.md?fbclid=IwAR1BFK5X1XdUpx_QXoiqyfzYWdNAXJPcg8Cf0ddv5T7sa2UrLUvymj1J8G4">OPT-175B LICENSE AGREEMENT</a></td>
459+
<td>❌</td>
460+
<td>No development relating to surveillance research and military, no harm the public interest of society</td>
461+
<td>Public</td>
462+
<td>RoBERTa corpus, the Pile, PushShift.io Reddit</td>
463+
</tr>
464+
<tr>
465+
<td>---&gt; OPT-IML</td>
466+
<td><a href="https://github.com/facebookresearch/metaseq/blob/main/projects/OPT/MODEL_LICENSE.md?fbclid=IwAR1BFK5X1XdUpx_QXoiqyfzYWdNAXJPcg8Cf0ddv5T7sa2UrLUvymj1J8G4">OPT-175B LICENSE AGREEMENT</a></td>
467+
<td>❌</td>
468+
<td>same to OPT</td>
469+
<td>Public</td>
470+
<td>OPT corpus, Extended version of Super-NaturalInstructions</td>
471+
</tr>
472+
<tr>
473+
<td>YaLM</td>
474+
<td>Apache 2.0</td>
475+
<td>✅</td>
476+
<td> </td>
477+
<td>Unspecified</td>
478+
<td>Pile, Teams collected Texts in Russian</td>
479+
</tr>
480+
<tr>
481+
<td>BLOOM</td>
482+
<td><a href="https://bigscience.huggingface.co/blog/the-bigscience-rail-license">The BigScience RAIL License</a></td>
483+
<td>✅</td>
484+
<td>No use of generating verifiably false information with the purpose of harming others; <br/>content without expressly disclaiming that the text is machine generated</td>
485+
<td>Public</td>
486+
<td>ROOTS corpus (Lauren¸con et al., 2022)</td>
487+
</tr>
488+
<tr>
489+
<td>---&gt; BLOOMZ</td>
490+
<td><a href="https://bigscience.huggingface.co/blog/the-bigscience-rail-license">The BigScience RAIL License</a></td>
491+
<td>✅</td>
492+
<td>same to BLOOM</td>
493+
<td>Public</td>
494+
<td>ROOTS corpus, xP3</td>
495+
</tr>
496+
<tr>
497+
<td>Galactica</td>
498+
<td><a href="https://github.com/paperswithcode/galai/blob/main/LICENSE-MODEL.md">CC BY-NC 4.0</a></td>
499+
<td>❌</td>
500+
<td> </td>
501+
<td>N/A</td>
502+
<td>The Galactica Corpus</td>
503+
</tr>
504+
<tr>
505+
<td>LLaMA</td>
506+
<td><a href="https://docs.google.com/forms/d/e/1FAIpQLSfqNECQnMkycAp2jP4Z9TFX0cGR4uf7b_fBxjY_OjhJILlKGA/viewform">Non-commercial bespoke license</a></td>
507+
<td>❌</td>
508+
<td>No development relating to surveillance research and military, no harm the public interest of society</td>
509+
<td>Public</td>
510+
<td>CommonCrawl, C4, Github, Wikipedia, etc.</td>
511+
</tr>
512+
<tr>
513+
<td>---&gt; Alpaca</td>
514+
<td>CC BY NC 4.0</td>
515+
<td>❌</td>
516+
<td> </td>
517+
<td>CC BY NC 4.0, Subject to terms of Use of the data generated by OpenAI</td>
518+
<td>LLaMA corpus, Self-Instruct</td>
519+
</tr>
520+
<tr>
521+
<td>---&gt; Vicuna</td>
522+
<td>CC BY NC 4.0</td>
523+
<td>❌</td>
524+
<td> </td>
525+
<td>Subject to terms of Use of the data generated by OpenAI; <br/>Privacy Practices of ShareGPT</td>
526+
<td>LLaMA corpus, 70K conversations from <a href="http://sharegpt.com/">ShareGPT.com</a></td>
527+
</tr>
528+
<tr>
529+
<td>---&gt; GPT4ALL</td>
530+
<td>GPL Licensed LLaMa</td>
531+
<td>❌</td>
532+
<td> </td>
533+
<td>Public</td>
534+
<td><a href="https://huggingface.co/datasets/nomic-ai/gpt4all_prompt_generations">GPT4All dataset</a></td>
535+
</tr>
536+
<tr>
537+
<td>OpenLLaMA</td>
538+
<td>Apache 2.0</td>
539+
<td>✅</td>
540+
<td> </td>
541+
<td>Public</td>
542+
<td><a href="https://www.together.xyz/blog/redpajama">RedPajama</a></td>
543+
</tr>
544+
<tr>
545+
<td>CodeGeeX</td>
546+
<td><a href="https://github.com/THUDM/CodeGeeX/blob/main/MODEL_LICENSE">The CodeGeeX License</a></td>
547+
<td>❌</td>
548+
<td>No use for illegal purposes or military research</td>
549+
<td>Public</td>
550+
<td>Pile, CodeParrot, etc.</td>
551+
</tr>
552+
<tr>
553+
<td>StarCoder</td>
554+
<td><a href="https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement">BigCode OpenRAIL-M v1 license</a></td>
555+
<td>✅</td>
556+
<td>No use of generating verifiably false information with the purpose of harming others; <br/>content without expressly disclaiming that the text is machine generated</td>
557+
<td>Public</td>
558+
<td><a href="https://arxiv.org/pdf/2211.15533.pdf">The Stack</a></td>
559+
</tr>
560+
</tbody></table>
561+
276562
## Star History
277563

278564
[![Star History Chart](https://api.star-history.com/svg?repos=Mooler0410/LLMsPracticalGuide&type=Date)](https://star-history.com/#Mooler0410/LLMsPracticalGuide&Date)

0 commit comments

Comments
 (0)