Skip to content

Add Float8 support for AQT tensor parallel #1003

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Oct 4, 2024
Merged

Add Float8 support for AQT tensor parallel #1003

merged 1 commit into from
Oct 4, 2024

Conversation

jainapurva
Copy link
Contributor

@jainapurva jainapurva commented Oct 3, 2024 •
edited
Loading

Add support to AQT Tensor Parallel for Float8 Weight Only

  - Added op support for aten.slice.Tensor
  - Added kernel in AQT dispatcher for float8-weight-only
  - Added AQT tensor parallel test case, can be run on H100

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Oct 3, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/1003

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit bcb82c4 with merge base 9229df9 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 3, 2024
@jainapurva jainapurva mentioned this pull request Oct 3, 2024
Closed
@jainapurva jainapurva force-pushed the float8_tensor_parallel branch from bcbec9e to f85ee95 Compare October 4, 2024 00:13
@jainapurva jainapurva force-pushed the float8_tensor_parallel branch from f85ee95 to 39dc48d Compare October 4, 2024 19:04
@jainapurva jainapurva marked this pull request as ready for review October 4, 2024 19:05
@jainapurva jainapurva linked an issue Oct 4, 2024 that may be closed by this pull request
Tensor Parallelism Support for AffineQuantizedTensor #988
Open
6 tasks
@jainapurva jainapurva requested a review from jerryzh168 October 4, 2024 19:24
@jainapurva jainapurva force-pushed the float8_tensor_parallel branch from 39dc48d to b18c876 Compare October 4, 2024 21:18
@jainapurva jainapurva force-pushed the float8_tensor_parallel branch from b18c876 to bcb82c4 Compare October 4, 2024 21:47
@jainapurva jainapurva merged commit 9e2a253 into main Oct 4, 2024
17 checks passed
melvinebenezer pushed a commit to melvinebenezer/ao that referenced this pull request Oct 7, 2024
@jainapurva @melvinebenezer
Add Float8 support for AQT tensor parallel (pytorch#1003)
b6fde7e
@jainapurva jainapurva mentioned this pull request Oct 7, 2024
Open
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jerryzh168 jerryzh168 jerryzh168 approved these changes

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Tensor Parallelism Support for AffineQuantizedTensor
3 participants
@jainapurva @jerryzh168 @facebook-github-bot