SFTTrainer crashes on Qwen3.5 because forward is functools.partial

[Eraly-ml]

SFTTrainer crashes on Qwen3.5 multimodal model because forward is a functools.partial

Description

trl.SFTTrainer crashes during initialization when used with the official Qwen/Qwen3.5-0.8B checkpoint from Transformers.

The failure occurs before training starts, while TRL tries to apply its chunked cross-entropy LM-head patch. The patch assumes that model.forward is a bound method and accesses original_forward.__func__, but Qwen3.5's decorated multimodal forward is a functools.partial and does not expose __func__.

This prevents using SFTTrainer with Qwen3_5ForConditionalGeneration, even for text-only training with the vision tower frozen.

Minimal reproduction

import torch
from datasets import load_dataset
from transformers import AutoTokenizer, Qwen3_5ForConditionalGeneration
from peft import LoraConfig
from trl import SFTConfig, SFTTrainer

model_id = "Qwen/Qwen3.5-0.8B"

model = Qwen3_5ForConditionalGeneration.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto",
)

# Text-only experiment: freeze vision explicitly.
for parameter in model.model.visual.parameters():
    parameter.requires_grad = False

model.config.use_cache = False

tokenizer = AutoTokenizer.from_pretrained(model_id)
dataset = load_dataset(
    "Eraly-ml/qwen35-kazakh-instruction-pilot",
    split="train",
)

# The dataset contains a `messages` column. Convert it to a text column.
def format_example(example):
    return {
        "text": tokenizer.apply_chat_template(
            example["messages"],
            tokenize=False,
            add_generation_prompt=False,
            enable_thinking=False,
        )
    }

dataset = dataset.map(format_example)

peft_config = LoraConfig(
    r=16,
    lora_alpha=32,
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM",
    target_modules=[
        "q_proj", "k_proj", "v_proj", "o_proj",
        "out_proj",
        "gate_proj", "up_proj", "down_proj",
    ],
)

args = SFTConfig(
    output_dir="./qwen35-test",
    num_train_epochs=1,
    per_device_train_batch_size=1,
    max_length=512,
    report_to="none",
)

trainer = SFTTrainer(
    model=model,
    args=args,
    train_dataset=dataset,
    processing_class=tokenizer,
    peft_config=peft_config,
)

Actual behavior

The constructor fails before trainer.train():

AttributeError: 'functools.partial' object has no attribute '__func__'

Relevant traceback:

trl/trainer/sft_trainer.py
    _patch_chunked_ce_lm_head(...)

trl/trainer/sft_trainer.py
    _chunked_ce_forward.__signature__ = inspect.signature(
        original_forward.__func__
    )

AttributeError: 'functools.partial' object has no attribute '__func__'

The failure occurs in the following path:

SFTTrainer.__init__
  -> _patch_chunked_ce_lm_head
  -> inspect.signature(original_forward.__func__)
  -> AttributeError

Expected behavior

SFTTrainer should either:

1. support Qwen3_5ForConditionalGeneration without patching its forward method incorrectly; or
2. detect that the forward callable is a functools.partial and inspect it without assuming __func__; or
3. provide a compatibility fallback that disables the chunked CE patch for unsupported multimodal model classes.

For example, the patch could handle both bound methods and callable objects:

forward = original_forward

if hasattr(forward, "__func__"):
    signature_source = forward.__func__
else:
    signature_source = forward

_chunked_ce_forward.__signature__ = inspect.signature(signature_source)

The exact implementation may need to preserve the original multimodal forward signature, including arguments such as pixel_values, image_grid_thw, and video_grid_thw.

Environment

• Model: Qwen/Qwen3.5-0.8B
• Model class: Qwen3_5ForConditionalGeneration
• Dataset: Eraly-ml/qwen35-kazakh-instruction-pilot
• Dataset size: 2,308 examples
• Python: 3.12 (Kaggle)
• PyTorch: 2.10.0+cu128
• GPU: NVIDIA Tesla T4, 14.56 GB VRAM
• Transformers: latest version installed from PyPI during the run
• TRL: latest version installed from PyPI during the run
• PEFT: latest version installed from PyPI during the run

The exact package versions can be captured with:

import torch, transformers, trl, peft
print("torch", torch.__version__)
print("transformers", transformers.__version__)
print("trl", trl.__version__)
print("peft", peft.__version__)

Additional context

The model loads successfully and PEFT can inject LoRA adapters after removing an unrelated old torchao package from the Kaggle environment. The failure is specific to the TRL SFTTrainer initialization path.

Using the lower-level transformers.Trainer together with peft.get_peft_model(...) avoids this error and allows the model to train. However, this workaround loses the higher-level SFT functionality provided by TRL.

The model's vision tower was frozen and no image inputs were provided. Therefore, this issue also affects text-only SFT use cases with Qwen3.5 multimodal checkpoints.

Suggested labels

• bug
• transformers integration
• multimodal
• Qwen3.5

[qgallouedec]

Please try with the recommend way to use SFT:

import torch
from datasets import load_dataset
from peft import LoraConfig
from trl import SFTConfig, SFTTrainer

dataset = load_dataset("Eraly-ml/qwen35-kazakh-instruction-pilot", split="train")

def add_chat_template_kwargs(example):
    return {"chat_template_kwargs": {"enable_thinking": False}}

dataset = dataset.map(add_chat_template_kwargs)

peft_config = LoraConfig(
    r=16,
    lora_alpha=32,
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM",
    target_modules=[
        "q_proj", "k_proj", "v_proj", "o_proj",
        "out_proj",
        "gate_proj", "up_proj", "down_proj",
    ],
)

args = SFTConfig(
    output_dir="./qwen35-test",
    num_train_epochs=1,
    per_device_train_batch_size=1,
    max_length=512,
    report_to="none",
)

trainer = SFTTrainer(
    model="Qwen/Qwen3.5-0.8B",
    args=args,
    train_dataset=dataset,
    peft_config=peft_config,
)