Skip to content

Commit 7f3b7dc

Browse files
iamzainhudafacebook-github-bot
authored andcommitted
fix shardedtensor init (#2586)
Summary: Pull Request resolved: #2586 we have global metadata for shardedtensor to use for init - we can avoid allgather altogether. Reviewed By: kausv Differential Revision: D66394789 fbshipit-source-id: 6edde29a711110f0588c5040db0e330d1b796d1b
1 parent e6fe5de commit 7f3b7dc

File tree

1 file changed

+3
-3
lines changed

1 file changed

+3
-3
lines changed

torchrec/distributed/embeddingbag.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -953,9 +953,9 @@ def _initialize_torch_state(self) -> None: # noqa
953953
)
954954

955955
self._model_parallel_name_to_sharded_tensor[table_name] = (
956-
ShardedTensor._init_from_local_shards(
957-
local_shards,
958-
self._name_to_table_size[table_name],
956+
ShardedTensor._init_from_local_shards_and_global_metadata(
957+
local_shards=local_shards,
958+
sharded_tensor_metadata=metadata,
959959
process_group=(
960960
self._env.sharding_pg
961961
if isinstance(self._env, ShardingEnv2D)

0 commit comments

Comments
 (0)