language-models.md

Language Models

• Grok-2 [Blog]
  • xAI
  • Grok-2 Beta was released on 2024/08/13.
• Gemma 2: Improving Open Language Models at a Practical Size (arXiv:2408.00118) [arXiv] [Code]
  • Gemma Team, Google DeepMind
  • Gemma 2
  • Models: https://www.kaggle.com/models/google/gemma
• The Llama 3 Herd of Models (arXiv:2407.21783) [arXiv] [Blog] [Code]
  • MetaAI
  • Llama 3
  • Models
    • Llama 3 8B: https://huggingface.co/meta-llama/Meta-Llama-3-8B
    • Llama 3 70B
    • Llama 3 405B
• Mixtral 8x7B (arXiv:2401.04088) [arXiv] [Blog] [Code]
  • Mistral AI
  • Mixtral 8x7B
  • Model: https://huggingface.co/mistralai/Mixtral-8x7B-v0.1
• Llama 2: Open Foundation and Fine-Tuned Chat Models (arXiv 2307.09288) [Paper] [Homepage]
  • Meta AI
  • Llama 2
  • Released with a permissive community license and is available for commercial use.
• LLaMA: Open and Efficient Foundation Language Models (arXiv 2302.13971) [Paper] [Code]
  • Meta AI
  • 6.7B, 13B, 32.5B, 65.2B
  • Open-access
• PaLM: Scaling Language Modeling with Pathways (JMLR 2023) [Paper] [PaLM API]
  • 540B; open access to PaLM APIs in March 2023.
• BLOOM: A 176B-Parameter Open-Access Multilingual Language Model (arXiv 2211.05100) [Paper] [Model] [Blog]
  • 176B
  • open-access
• OPT: Open Pre-trained Transformer Language Models (arXiv: 2205.01068) [Paper] [Code]
  • Meta AI
  • Range from 125M to 175B parameters.
  • Open-access