feat(inference): HfQuantizer integration for pre-quantized GPTQ/AWQ models #1954

Summary

Add support for loading pre-quantized models (GPTQ, AWQ) via HuggingFace's native AutoHfQuantizer framework, enabling layer-by-layer inference of production-optimized quantized models.

Background

Most practical open-source models for the RTX 4070 (8GB VRAM) are distributed pre-quantized by their creators (TheBloke, etc.). These use GPTQ or AWQ formats which are already 4-bit on disk and optimized for quality. This is more important than runtime quantization (#1943) for production use.

airllm integrates HfQuantizer by:

1. Reading quantization_config from the model's HuggingFace config
2. Creating a quantizer via AutoHfQuantizer.from_config(config, pre_quantized=True)
3. During layer loading, checking each parameter: hf_quantizer.check_quantized_param()
4. Creating quantized params in-place: hf_quantizer.create_quantized_param()

# In init_model()
quantization_config = getattr(config, "quantization_config", None)
if quantization_config is not None:
    self.hf_quantizer = AutoHfQuantizer.from_config(quantization_config, pre_quantized=True)
    self.hf_quantizer.preprocess_model(model=self.model, device_map=device_map)

# In move_layer_to_device() — per parameter
if self.hf_quantizer.check_quantized_param(self.model, state_dict[param_name], param_name, ...):
    self.hf_quantizer.create_quantized_param(self.model, state_dict[param_name], param_name, ...)

Key Files

• autobot-backend/llm_interface_pkg/optimization/quantization.py — HfQuantizer integration
• autobot-backend/llm_interface_pkg/providers/ — wire into local model providers
• autobot-backend/llm_interface_pkg/optimization/router.py — detect and route pre-quantized models

Acceptance Criteria

• Auto-detect pre-quantized models from config's quantization_config
• GPTQ models load and run correctly layer-by-layer
• AWQ models load and run correctly layer-by-layer
• Per-parameter quantized handling during layer loading
• Meta device eviction works with quantized params (per-param path)
• Benchmark: GPTQ 7B model on RTX 4070 — latency, memory, output quality

Related Issues

• feat(optimization): wire block-wise weight quantization in OptimizationRouter #1943 (block-wise quantization — runtime quantization, this is pre-quantized loading)
• feat(optimization): layer-by-layer inference mode for batch/offline processing #1946 (layer-by-layer inference — pre-quantized models use the same layer pipeline)