GGUF Dequantization: Row misalignment in dequantized tensors - only first column matches original

[wzqww23]

I'm experiencing an issue where dequantized GGUF tensors don't match the original PyTorch model tensors, except for the first column. I have use gguf.quants.quantize

Environment

Model: Llama 3.2 1B Instruct (Q8_0 quantization)
GGUF file: unsloth/Llama-3.2-1B-Instruct-GGUF from huggingface
Original model loaded with torch_dtype=torch.float32

Code

python

# Loading original PyTorch model
torch_model = modeling_llama.LlamaModel.from_pretrained(
    "meta-llama/Llama-3.2-1B-Instruct",
    torch_dtype=torch.float32
).eval()

# Loading GGUF and dequantizing
reader = GGUFReader("/path/to/Llama-3.2-1B-Instruct-Q8_0.gguf")
target_tensor = None
for tensor in reader.tensors:
    if tensor.name == "blk.0.attn_k.weight":
        target_tensor = tensor
        break

# Comparing tensors
t1 = torch.from_numpy(dequantized_tensor)  # GGUF dequantized
t2 = torch_model.state_dict()["layers.0.self_attn.k_proj.weight"]  # Original PyTorch

Issue
The tensors have the same shape (512, 2048), but the values don't align correctly:

python

Row 0 - ✅ MATCHES (mostly)
Dequantized: [ 0.0584,  0.1180,  0.0618, -0.0401, -0.0527]
Original   : [ 0.0581,  0.1177,  0.0615, -0.0396, -0.0522]

Row 1 - ❌ MISMATCHED  
Dequantized: [ 0.0851,  0.0252,  0.0070, -0.0226, -0.0061]
Original   : [-0.0123,  0.0742,  0.0576, -0.0190, -0.0264]

Row 2 - ❌ Row 1's dequantized values appear in Row 2 of original
Dequantized: [-0.0120,  0.0744,  0.0576, -0.0192, -0.0264]  
Original   : [-0.0771, -0.0386, -0.0023,  0.0212, -0.0136]

Expected Behavior

Dequantized GGUF tensors should closely match the original PyTorch model tensors across all rows and columns, not just the first column.