Skip to content

autoquant support for fp8 #715

Closed
Closed
@msaroufim

Description

@msaroufim

Recently came across this repo that's doing fp8 inference https://github.com/aredden/flux-fp8-api/blob/main/float8_quantize.py

It's getting popular enough that we should consider just making this a setting for autoquant

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions