Skip to content

Add document to describe the strategy of splitting parameter in distributed transpiler #11250

Closed
@velconia

Description

@velconia

Including three levels of details:

Why should we split parameters

  1. the benefits of splitting parameters

How we do split of parameters

  1. How to slice variables into blocks.
  2. How to split blocks into parameter servers.

A use case of training

  1. distributed training case with parameters split up

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions