Bump Int4WeightOnlyConfig version to 2

[jerryzh168]

Summary:
Current Int4WeightOnlyConfig has version 1 and 2, and default is 1, this PR

• changes the default to 2 and made modification to callsites.
• For the Int4WeightOnlyConfig using explicit version 2, we removed the version=2 since now default is 2
• For the Int4WeightOnlyConfig that's using the old configuration, we added explicit version=1, we can migrate the callsite to use the version 2 separately (note this is done in Add version=1 for calls to int4 weight only config #2958)
• also added deprecation warning in
  • quant_api for v1 path of Int4WeightOnlyConfig
  • different layouts (TensorCoreTiledLayout, MarlinSparseLayout, Int4CPULayout and Int4XPULayout) for quantized checkpoints with v1 config
• For READMEs we migrate the usage to version 2 directly
• Also added deprecation warning testing for v1
  • single linear: https://huggingface.co/torchao-testing/single-linear-Int4WeightOnlyConfig-v1-0.14.dev (testing checkpoint only)
  • opt-125m: https://huggingface.co/torchao-testing/opt-125m-Int4WeightOnlyConfig-v1-0.14.dev (testing both checkpoint and config)

Deprecation Note:

We updated the implementation for int4 Tensor, so bumps the default version from 1 to 2 for these two configs.

from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "torchao-testing/opt-125m-Int4WeightOnlyConfig-v1-0.14.dev"
quantized_model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="bfloat16",
    device_map="cuda",
)

/data/users/jerryzh/ao/torchao/core/config.py:250: UserWarning: Stored version is not the same as current default version of the config: stored_version=1, current_default_version=2, please check the deprecation warning
  warnings.warn(
/data/users/jerryzh/ao/torchao/dtypes/uintx/tensor_core_tiled_layout.py:241: UserWarning: Models quantized with version 1 of Int4WeightOnlyConfig is deprecated and will no longer be supported in a future release, please upgrade torchao and quantize again, or download a newer torchao checkpoint, see https://github.com/pytorch/ao/issues/2948 for more details
  warnings.warn(

Suggestion: upgrade torchao to 0.14 and later and generate the checkpoint again:

quantize_(model, Int4WeightOnlyConfig(group_size=128))

Or download the checkpoint again (please let us know if the checkpoint is not updated)

Please see #2948 for more details around the deprecation.

Test Plan:
Regression tests:
python test/dtypes/test_affine_quantized.py
python test/quantization/test_quant_api.py
python test/quantization/quantize_/workflows/int4/test_int4_marlin_sparse_tensor.py
python test/quantization/quantize_/workflows/int4/test_int4_opaque_tensor.py
python test/quantization/quantize_/workflows/int4/test_int4_plain_int32_tensor.py
python test/quantization/quantize_/workflows/int4/test_int4_preshuffled_tensor.py
python test/quantization/quantize_/workflows/int4/test_int4_tensor.py
python test/quantization/quantize_/workflows/int4/test_int4_tile_packed_to_4d_tensor.py
python test/integration/test_load_and_run_checkpoint.py

Reviewers:

Subscribers:

Tasks:

Tags:

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/2949

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit d2168f2 with merge base c452495 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[metascroy]

It's called int4_packing_format now, no?

[jerryzh168]

yeah that's true, I have updated locally, will push change together with other things

[jerryzh168]

just found we also need to update packing_format to int4_packing_format I have made change locally, can push these changes before land.

[metascroy]

Should you import to fbcode to see if you break any internal tests?

[facebook-github-bot]

@jerryzh168 has imported this pull request. If you are a Meta employee, you can view this in D81985661.

[jerryzh168]

looks like there are some conflicts in importing, I'll unlink and merge for now, will rely on diff train