Modifying interaction layer to include 2 MLPs in DLRM (#382) #1
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
fbshipit-source-id: 21a7f5abf0232187fdb76109e40a9ff571ef40bb
…agePlugin Summary: We add a `remove` function to the `FSStoragePlugin` in order to implement a delete API for TorchSnapshot. Similar functions for `GCSStoragePlugin` and `S3StoragePlugin` and their integration with `Snapshot` will be added in subsequent diffs. Reviewed By: yifuwang Differential Revision: D36948787 fbshipit-source-id: fc37336e5738a9122bb72f68afe192491faceb13
facebook-github-bot
added
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D35861688 |
Summary: Pull Request resolved: pytorch/torchsnapshot#1 X-link: facebookresearch/recipes#26 X-link: pytorch/torchrec#382 X-link: facebookresearch/dlrm#242 This diff adds 2 MLPs to the interaction layer in DLRM for MLPerf update. New DLRM module called DLRMV2 can be realized by --dlrmv2 argument. Additional arguments for the interaction MLPs are --interaction_branch1_layer_sizes and --interaction_branch2_layer_sizes to pass in the MLP sizes. The output dimension of the interaction MLPs must be a multiple of the embedding dimension. DLRMTrain now takes in a DLRM/DLRMV2 module at construction time. Reviewed By: colin2328, samiwilf Differential Revision: D35861688 fbshipit-source-id: bbeec2ac28b711458af67db90c7b8c8e5b6f081e
narayanan2004
added a commit
to narayanan2004/recipes
that referenced
this pull request
Jun 9, 2022
Summary: X-link: pytorch/torchsnapshot#1 Pull Request resolved: facebookresearch#26 X-link: pytorch/torchrec#382 X-link: facebookresearch/dlrm#242 This diff adds 2 MLPs to the interaction layer in DLRM for MLPerf update. New DLRM module called DLRMV2 can be realized by --dlrmv2 argument. Additional arguments for the interaction MLPs are --interaction_branch1_layer_sizes and --interaction_branch2_layer_sizes to pass in the MLP sizes. The output dimension of the interaction MLPs must be a multiple of the embedding dimension. DLRMTrain now takes in a DLRM/DLRMV2 module at construction time. Reviewed By: colin2328, samiwilf Differential Revision: D35861688 fbshipit-source-id: 8f730c3b23bea170bcf7df39c420726a0092c1a5
narayanan2004
added a commit
to narayanan2004/dlrm
that referenced
this pull request
Jun 9, 2022
…ch#1) Summary: X-link: pytorch/torchsnapshot#1 X-link: facebookresearch/recipes#26 X-link: pytorch/torchrec#382 Pull Request resolved: facebookresearch#242 This diff adds 2 MLPs to the interaction layer in DLRM for MLPerf update. New DLRM module called DLRMV2 can be realized by --dlrmv2 argument. Additional arguments for the interaction MLPs are --interaction_branch1_layer_sizes and --interaction_branch2_layer_sizes to pass in the MLP sizes. The output dimension of the interaction MLPs must be a multiple of the embedding dimension. DLRMTrain now takes in a DLRM/DLRMV2 module at construction time. Reviewed By: colin2328, samiwilf Differential Revision: D35861688 fbshipit-source-id: a371551c9a73fc1e3339423bc4443c6d077990ef
narayanan2004
force-pushed
the
export-D35861688
branch
from
831dfe5 to
455acc5
Compare
|
This pull request was exported from Phabricator. Differential Revision: D35861688 |
narayanan2004
added a commit
to narayanan2004/torchrec
that referenced
this pull request
Jun 9, 2022
Summary: X-link: pytorch/torchsnapshot#1 X-link: facebookresearch/recipes#26 Pull Request resolved: pytorch#382 X-link: facebookresearch/dlrm#242 This diff adds 2 MLPs to the interaction layer in DLRM for MLPerf update. New DLRM module called DLRMV2 can be realized by --dlrmv2 argument. Additional arguments for the interaction MLPs are --interaction_branch1_layer_sizes and --interaction_branch2_layer_sizes to pass in the MLP sizes. The output dimension of the interaction MLPs must be a multiple of the embedding dimension. DLRMTrain now takes in a DLRM/DLRMV2 module at construction time. Reviewed By: colin2328, samiwilf Differential Revision: D35861688 fbshipit-source-id: e8d4e7cd45260f4d229553242b6ea48068f5dda9
facebook-github-bot
pushed a commit
that referenced
this pull request
Jun 9, 2022
Summary: Pull Request resolved: #1 X-link: facebookresearch/recipes#26 X-link: pytorch/torchrec#382 X-link: facebookresearch/dlrm#242 This diff adds 2 MLPs to the interaction layer in DLRM for MLPerf update. New DLRM module called DLRMV2 can be realized by --dlrmv2 argument. Additional arguments for the interaction MLPs are --interaction_branch1_layer_sizes and --interaction_branch2_layer_sizes to pass in the MLP sizes. The output dimension of the interaction MLPs must be a multiple of the embedding dimension. DLRMTrain now takes in a DLRM/DLRMV2 module at construction time. Reviewed By: colin2328, samiwilf Differential Revision: D35861688 fbshipit-source-id: 69b61f8c620d81a92155f4a2824170b2ade34190
facebook-github-bot
pushed a commit
to facebookresearch/recipes
that referenced
this pull request
Jun 9, 2022
Summary: X-link: pytorch/torchsnapshot#1 Pull Request resolved: #26 X-link: pytorch/torchrec#382 X-link: facebookresearch/dlrm#242 This diff adds 2 MLPs to the interaction layer in DLRM for MLPerf update. New DLRM module called DLRMV2 can be realized by --dlrmv2 argument. Additional arguments for the interaction MLPs are --interaction_branch1_layer_sizes and --interaction_branch2_layer_sizes to pass in the MLP sizes. The output dimension of the interaction MLPs must be a multiple of the embedding dimension. DLRMTrain now takes in a DLRM/DLRMV2 module at construction time. Reviewed By: colin2328, samiwilf Differential Revision: D35861688 fbshipit-source-id: 69b61f8c620d81a92155f4a2824170b2ade34190
facebook-github-bot
pushed a commit
to pytorch/torchrec
that referenced
this pull request
Jun 9, 2022
Summary: X-link: pytorch/torchsnapshot#1 X-link: facebookresearch/recipes#26 Pull Request resolved: #382 X-link: facebookresearch/dlrm#242 This diff adds 2 MLPs to the interaction layer in DLRM for MLPerf update. New DLRM module called DLRMV2 can be realized by --dlrmv2 argument. Additional arguments for the interaction MLPs are --interaction_branch1_layer_sizes and --interaction_branch2_layer_sizes to pass in the MLP sizes. The output dimension of the interaction MLPs must be a multiple of the embedding dimension. DLRMTrain now takes in a DLRM/DLRMV2 module at construction time. Reviewed By: colin2328, samiwilf Differential Revision: D35861688 fbshipit-source-id: 69b61f8c620d81a92155f4a2824170b2ade34190
facebook-github-bot
pushed a commit
to facebookresearch/dlrm
that referenced
this pull request
Jun 9, 2022
Summary: X-link: pytorch/torchsnapshot#1 X-link: facebookresearch/recipes#26 X-link: pytorch/torchrec#382 Pull Request resolved: #242 This diff adds 2 MLPs to the interaction layer in DLRM for MLPerf update. New DLRM module called DLRMV2 can be realized by --dlrmv2 argument. Additional arguments for the interaction MLPs are --interaction_branch1_layer_sizes and --interaction_branch2_layer_sizes to pass in the MLP sizes. The output dimension of the interaction MLPs must be a multiple of the embedding dimension. DLRMTrain now takes in a DLRM/DLRMV2 module at construction time. Reviewed By: colin2328, samiwilf Differential Revision: D35861688 fbshipit-source-id: 69b61f8c620d81a92155f4a2824170b2ade34190
allenfengjr
pushed a commit
to allenfengjr/dlrm_comp
that referenced
this pull request
Jul 2, 2023
Summary: X-link: pytorch/torchsnapshot#1 X-link: facebookresearch/recipes#26 X-link: pytorch/torchrec#382 Pull Request resolved: facebookresearch/dlrm#242 This diff adds 2 MLPs to the interaction layer in DLRM for MLPerf update. New DLRM module called DLRMV2 can be realized by --dlrmv2 argument. Additional arguments for the interaction MLPs are --interaction_branch1_layer_sizes and --interaction_branch2_layer_sizes to pass in the MLP sizes. The output dimension of the interaction MLPs must be a multiple of the embedding dimension. DLRMTrain now takes in a DLRM/DLRMV2 module at construction time. Reviewed By: colin2328, samiwilf Differential Revision: D35861688 fbshipit-source-id: 69b61f8c620d81a92155f4a2824170b2ade34190
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
X-link: pytorch/torchrec#382
X-link: facebookresearch/dlrm#242
This diff adds 2 MLPs to the interaction layer in DLRM for MLPerf update. New DLRM module called DLRMV2 can be realized by --dlrmv2 argument. Additional arguments for the interaction MLPs are --interaction_branch1_layer_sizes and --interaction_branch2_layer_sizes to pass in the MLP sizes. The output dimension of the interaction MLPs must be a multiple of the embedding dimension.
DLRMTrain now takes in a DLRM/DLRMV2 module at construction time.
Reviewed By: colin2328, samiwilf
Differential Revision: D35861688