[model_free_ptq] Add pathway for day-zero weight quantization support

[kylesayrs]

Purpose

• Create a pathway which can quantize model weights without needing a model definition or the use of a calibration pipeline. Such a pathway provides fast and reliable support for models which:
  • Do not have a HF model definition yet
  • Have complications with sequential pipelines (very large vision towers, tracing failure, long calibration runtime)

Usage

model_free_ptq(
    model_stub="meta-llama/Llama-3.2-1B-Instruct",
    save_directory="Llama-3.2-1B-Instruct-FP8_block",
    scheme="FP8_BLOCK",
    ignore=["model.embed_tokens", "lm_head"],
    max_workers=15,
    device="cuda:0",
):

Testing

• Added test_model_free_ptq_matches_oneshot which tests that saved tensors and configs exactly match between model_free_ptq and oneshot entrypoints for the same arguments. This test takes about 10 seconds to run.

Future Extensions

• Mixed-precision quantization (multiple recipes/targets)
• Multi-GPU support (work is already parallelized by threads, but if GPU is the bottleneck we can split the work across GPUs)
• Multi-process support (is python processing is the bottleneck, we can replace multithreading with multiprocessing)

[github-actions]

👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review.

Note: This is required to complete the testing suite, please only add the label once the PR is code complete and local testing has been performed.