Has anyone tried DM0 fine-tuning?

[ywangwxd]

I want to fine-tune its action expert but freeze the other parts.
I have encountered a lot of problems.

[hbzfeng]

Could you please share the specific error or problem you encountered, along with your training setting?

For example: the training config/script, checkpoint used, which modules are frozen/unfrozen, and the full error log.

[ywangwxd]

Could you please share the specific error or problem you encountered, along with your training setting?

For example: the training config/script, checkpoint used, which modules are frozen/unfrozen, and the full error log.

First of all, I have not found the script of fine-tuning DM0, only fine-tuning P0.
Can you help me to clarify this? Am I supposed to implement this script by myself?

Since this codebase was published by Dexmal, I thought it would firstly support DM0 fine-tuning :-).

[dexmal2025]

Here is the DM0 Tutorial: https://github.com/dexmal/dexbotic/blob/main/docs/DM0.md. We also provide two fine-tuning scripts of DM0 on Libero simulator (https://github.com/dexmal/dexbotic/blob/main/playground/benchmarks/libero/libero_dm0.py) and RoboChallenge Table30 (https://github.com/dexmal/dexbotic/blob/main/playground/benchmarks/table30/dm0_stack_bowls.py).

[ywangwxd]

Here is the DM0 Tutorial: https://github.com/dexmal/dexbotic/blob/main/docs/DM0.md. We also provide two fine-tuning scripts of DM0 on Libero simulator (https://github.com/dexmal/dexbotic/blob/main/playground/benchmarks/libero/libero_dm0.py) and RoboChallenge Table30 (https://github.com/dexmal/dexbotic/blob/main/playground/benchmarks/table30/dm0_stack_bowls.py).

Does this script support light-weight fine-tuning by freezing some heavy components, e.g., LLM, vision tower?

Since I found the notes on GPU requirements were very high.

We recommend using 8 × NVIDIA A100/H100 GPUs for training. If you are using 8 × RTX 4090, please use the configuration file scripts/deepspeed/zero3_offload.json to reduce GPU memory utilization. Normalization statistics are automatically computed before the first training run if not already cached.

[hbzfeng]

DM0 fine-tuning is supported, but the current DM0 example scripts are mainly set up for full-parameter fine-tuning. We have not yet exposed ready-to-use freeze flags for the LLM / vision tower in the DM0 config.

You can implement lightweight fine-tuning by modifying the DM0ModelConfig.build_model() method in your DM0 experiment script, for example playground/benchmarks/libero/libero_dm0.py or playground/benchmarks/table30/dm0_stack_bowls.py:

def build_model(self) -> DM0ForCausalLM:
    model = DM0ForCausalLM.from_pretrained(self.model_name_or_path)

    # Freeze heavy modules; action-related modules remain trainable by default.
    model.model.llm.requires_grad_(False)
    model.model.mm_vision_tower.requires_grad_(False)
    model.model.mm_projector.requires_grad_(False)

    return model

The optimizer only collects parameters with requires_grad=True, so this should work as long as it is done before trainer / optimizer initialization. We may add a cleaner config-level interface for this in the future. We are also planning to provide a LoRA-based training option in the near future to reduce hardware requirements.

[cuiyingd]

My two cents: maybe we can add config flags to freeze DM0 backbone modules (LLM / vision). This allows users to enable lightweight fine‑tuning directly via config, without modifying scripts.