Skip to content

[KNOWN BUG] gguf + GPU AOTI Inference bug due to PT version, fix in progress #1423

New issue
New issue
Closed
Closed
[KNOWN BUG] gguf + GPU AOTI Inference bug due to PT version, fix in progress#1423
Assignees
Jack-Khuu
Labels
CI InfraIssues related to CI infrastructure and setupCompile / AOTIIssues related to AOT Inductor and torch compileKnown GapsThese are known Gaps/Issues/Bug items in torchchatQuantizationIssues related to Quantization or torchaobugSomething isn't workingtriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate module
@Jack-Khuu

Description

@Jack-Khuu
Jack-Khuu
opened on Dec 14, 2024

🐛 Describe the bug

As of #1367, torchchat/main is failing 3-5 CI jobs related to GPU AOTI inference and GGUF inference

GPU AOTI inference will be fixed with a pinbump to pytorch/pytorch#143236
GGUF AO bug is being addressed in #1404

Versions

bb72b09

Metadata

Metadata

Assignees

Labels

CI InfraIssues related to CI infrastructure and setupCompile / AOTIIssues related to AOT Inductor and torch compileKnown GapsThese are known Gaps/Issues/Bug items in torchchatQuantizationIssues related to Quantization or torchaobugSomething isn't workingtriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate module

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions