Skip to content

dict device_map bug on RTN mode #890

@wenhuach21

Description

@wenhuach21

from auto_round import AutoRound
model_name = "/models/Qwen3-8B"
device_map = {"up_proj":0,"down_proj":1}

ar = AutoRound(model=model_name, scheme="W4A16", iters=0, nsamples=1, device_map=device_map)
model, layer_config = ar.quantize()

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions