New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Question] 关于第三阶段训练loss #66

Open

rangmiao opened this issue Mar 19, 2024 · 0 comments

rangmiao commented Mar 19, 2024

Question

作者你好，很感谢你们的工作！我在复现MoE-phi2X4-Top2 时，在第三阶段moe_loss 一直在16附近波动，想问下这是否正常

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment