Enable Int4WeightOnlyGPTQQuantizer on Intel GPU.

[xiaowangintel]

Following pytorch/pytorch#153019 requests, we enable int4wo-GPTQ for Intel GPU in pytorch/ao after RTN ready. Currently, the implementation of int4wo-GPTQ uses the ZeroPointDomain.FLOAT and ZeroPointDomain.INT.

How to run int4wo-GPTQ with ZeroPointDomain.INT:

import copy
import gc
import tempfile
import unittest
from pathlib import Path

import torch

from torchao._models._eval import InputRecorder, TransformerEvalWrapper
from torchao.quantization.GPTQ import Int4WeightOnlyGPTQQuantizer

from torchao._models.llama.model import Transformer, prepare_inputs_for_model
from torchao._models.llama.tokenizer import get_tokenizer

from torchao.quantization.quant_primitives import (
    ZeroPointDomain,
)

precision = torch.bfloat16
device = "xpu"
checkpoint_path = Path("../gpt-fast/checkpoints/meta-llama/Llama-2-7b-chat-hf/model.pth")
model = Transformer.from_name(checkpoint_path.parent.name)
checkpoint = torch.load(str(checkpoint_path), mmap=True, weights_only=True)
model.load_state_dict(checkpoint, assign=True)
model = model.to(dtype=precision, device="cpu")
model.eval()
tokenizer_path = checkpoint_path.parent / "tokenizer.model"
assert tokenizer_path.is_file(), tokenizer_path
tokenizer = get_tokenizer( 
    tokenizer_path,
    "Llama-2-7b-chat-hf",
)
blocksize = 128
percdamp = 0.01
groupsize = 128
calibration_tasks = ["wikitext"]
calibration_limit = 1
calibration_seq_length = 100
input_prep_func = prepare_inputs_for_model
pad_calibration_inputs = False

inputs = (
    InputRecorder(
        tokenizer,
        calibration_seq_length,
        input_prep_func,
        pad_calibration_inputs,
        model.config.vocab_size,
        device="cpu",
    )
    .record_inputs(
        calibration_tasks,
        calibration_limit,
    )
    .get_inputs()
)

quantizer = Int4WeightOnlyGPTQQuantizer(
    blocksize,
    percdamp,
    groupsize,
    device=torch.device(device),
    zero_point_domain = ZeroPointDomain.INT,
)
model.setup_caches(max_batch_size=1, max_seq_length=calibration_seq_length)
model = quantizer.quantize(model, inputs).to(device)

How to run int4wo-GPTQ with ZeroPointDomain.FLOAT:

quantizer = Int4WeightOnlyGPTQQuantizer(
    blocksize,
    percdamp,
    groupsize,
    device=torch.device(device),
)

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/2200

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[liangan1]

Suggested change

	def linear_forward_int4_zero_domain(
	def linear_forward_int4_zero_point_domain_int(

[xiaowangintel]

Modify done

[liangan1]

Only use once. Suggest to remove this function.

[xiaowangintel]

Modify done

[liangan1]

This function is also used by RTN path, this changes may break RTN, Pls double check.

[xiaowangintel]

align all usages

[liangan1]

Suggested change

[liangan1]

@jerryzh168 Can you help to review this PR?

[liangan1]

@EikanWang