Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to do the model parallelism ? #27

Open
vvictor-lee opened this issue Mar 11, 2020 · 0 comments
Open

How to do the model parallelism ? #27

vvictor-lee opened this issue Mar 11, 2020 · 0 comments

Comments

@vvictor-lee
Copy link

Great Work !
I found that the code works well on multiple gpu training but only for the data parallelism.
However, it is hard to train a model with the classes more than 10 million or even more. In that case,
the model parallelism should solve the problem. I was doing the implementations but the diam softmax may seriously interfere the model parallelism. Could you give the solutions or any idea ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant