Search results

megatron-lm中的DistributedOptimizer类似deepspeed的zero2吗？梯度切分部分求大佬上源码。

Status: Open.

#71 In alibaba/Megatron-LLaMA;

· zhoushuang66 opened on Dec 12, 2024

执行hf转megatron格式报错了

Status: Open.

#70 In alibaba/Megatron-LLaMA;

· Lilypad97 opened on Oct 21, 2024

TypeError: perform_nocolor_split(): incompatible function arguments.

Status: Open.

#69 In alibaba/Megatron-LLaMA;

· vedantgoswami opened on Aug 14, 2024

When training BERT, ERROR: "AttributeError: 'FullTokenizer' object has no attribute 'save_pretrained' "

Status: Open.

#68 In alibaba/Megatron-LLaMA;

· yuzhiguo07 opened on Aug 14, 2024

如何断点续训

Status: Open.

#66 In alibaba/Megatron-LLaMA;

· MAxx8371 opened on Jul 4, 2024

No update for a long time

Status: Open.

#65 In alibaba/Megatron-LLaMA;

· dong-liuliu opened on Jun 21, 2024

Has llama2 GQA been supported yet?

Status: Open.

#64 In alibaba/Megatron-LLaMA;

· JiwenJ opened on May 26, 2024

请问有dingding群聊，或者微信群吗？可以沟通的

Status: Open.

#63 In alibaba/Megatron-LLaMA;

· felix0080 opened on May 24, 2024

Llama 3 Support

Status: Open.

#62 In alibaba/Megatron-LLaMA;

· john-theo opened on May 3, 2024

About batch_size

Status: Open.

#61 In alibaba/Megatron-LLaMA;

· tszslovewanpu opened on Mar 16, 2024

sh LLaMA2_7B_standalone.sh

Status: Open.

#60 In alibaba/Megatron-LLaMA;

· yangzhipeng1108 opened on Mar 14, 2024

请问是否支持从0训练一个小规模的LLaMA模型，如：1B

Status: Open.

#59 In alibaba/Megatron-LLaMA;

· liubo12 opened on Mar 1, 2024