GTC DLI workflow docs by peterd-NV · Pull Request #390 · isaac-sim/IsaacLab-Arena

peterd-NV · 2026-02-05T21:06:07Z

Summary

Add docs for GTC DLI course GR1 sequential static manipulation workflow

xyao-nv · 2026-02-06T06:48:22Z

I included all OSMO related cmds here. We can remove them later if you think it's too much. But at least for TME, they can follow the exact steps 1-5 to make sure at least closedloop works for each individual task. Then We can do the cross task training. Otherwise we will be suffering from not working trajectories. @alexmillane WDYT

xyao-nv · 2026-02-06T06:49:11Z

@peterd-NV in closedloop, the close door move rate metric is 0, which does not quite make sense to me. Can you take a look?

alexmillane

Looks great! Thanks for doing this!

alexmillane · 2026-02-06T12:25:31Z

docs/pages/example_workflows/locomanipulation/step_4_evaluation.rst

-.. todo::
-
-   Verifying Arena-GR00T on Blackwell architecture.
-


alexmillane · 2026-02-06T12:28:58Z

docs/pages/example_workflows/sequential_static_manipulation/index.rst

+   * - **Interop**
+     - Isaac Lab Teleop, Isaac Lab Mimic
+   * - **Scene**
+     - Kitchen environment


Maybe we better put Light Wheel Robocasa Kitchen to attribute LightWheel.

alexmillane · 2026-02-06T12:29:31Z

docs/pages/example_workflows/sequential_static_manipulation/index.rst

+   * - **Scene**
+     - Kitchen environment
+   * - **Objects**
+     - Refrigerator (articulated object)


object to place in refrigerator (rigid object)

alexmillane · 2026-02-06T12:58:51Z

docs/pages/example_workflows/sequential_static_manipulation/step_3_data_generation.rst

+Follow the instructions described on the CLI to mark subtask boundaries:
+
+Left Arm:
+1. **Pick & place completed:** Robot has completed the pick and place task


Perhaps it's made clear in the CLI, but I'm wondering whether we should specify where the user should press the key here. Does this mean that you press the button once the door is closed? How does that interact with the reset.

Adding this to docs

alexmillane · 2026-02-06T12:59:27Z

docs/pages/example_workflows/sequential_static_manipulation/step_3_data_generation.rst

+Step 3: Validate Generated Data (Optional)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+In order to validation the generated dataset, you can replay the generated data


nit: in order to validate

alexmillane · 2026-02-06T13:09:10Z

docs/pages/example_workflows/sequential_static_manipulation/step_4_policy_training.rst

+To verify the generated LeRobot dataset, you first need to compute statistics information about the dataset.
+This can be done by running the following command:
+
+.. code-block:: bash
+
+   python submodules/Isaac-GR00T/gr00t/data/stats.py $DATASET_DIR/ranch_bottle_into_fridge/ranch_bottle_into_fridge_generated_100/lerobot gr1_joint


I notice that the stats command alters some files. Does this have an affect on training? I wasn't running this when training things in Zurich. I'm wondering if that could have affected the performance?

stats.py added stats.json & rel_stats.json which I think it's used in dataloader for normalization. It's included in training but not replaying. I don't think it's a bug from stats ops itself, as it mostly copies from n1.5.

alexmillane · 2026-02-06T13:09:22Z

docs/pages/example_workflows/sequential_static_manipulation/step_4_policy_training.rst

+
+.. todo::
+   There is a bug in ISAAC-GR00T submodule gr00t/data/stats.py. PR is submmited and in review from Gear.
+   Usele gpu finetune command in the last step for now.


Use finetune command?

alexmillane · 2026-02-06T13:15:00Z

docs/pages/example_workflows/sequential_static_manipulation/step_4_policy_training.rst

+         --dataset_path=$DATASET_DIR/arena_gr1_manipulation_dataset_generated/lerobot \
+         --output_dir=$MODELS_DIR \
+         --modality_config_path=isaaclab_arena_gr00t/embodiments/gr1/gr1_arms_only_data_config.py \
+         --global_batch_size=24 \


shall we go with 96? In my testing this results in a lower training error, and doesn't affect the training time.

good catch. It was trained with 96 but I forgot to update the doc here

alexmillane · 2026-02-06T13:17:09Z

docs/pages/example_workflows/sequential_static_manipulation/step_5_evaluation.rst

+   python isaaclab_arena/evaluation/policy_runner.py \
+     --policy_type isaaclab_arena_gr00t.policy.gr00t_closedloop_policy.Gr00tClosedloopPolicy \
+     --policy_config_yaml_path isaaclab_arena_gr00t/policy/config/gr1_manip_ranch_bottle_gr00t_closedloop_config.yaml \
+     --num_steps 2000 \


To me it makes more sense that we run for a number of episodes. I'm adding that here: #383

What do you think?

I think it's gonna be a great idea once you merge it. It looks like it has issue for recording metrics yesterday I ran it.

…tation for closing door

…rena into peterd/gtc_dli_docs

xyao-nv force-pushed the peterd/gtc_dli_docs branch from 71f1e9a to dbe0bac Compare February 6, 2026 06:45

xyao-nv marked this pull request as ready for review February 6, 2026 06:45

alexmillane approved these changes Feb 6, 2026

View reviewed changes

peterd-NV and others added 11 commits February 6, 2026 10:49

add docs for gtc dli workflow

0e1725e

add data gen docs for dli course workflow

b4d256a

clean up step 1 env setup docs

60c9b01

add env visuals

98a523d

Add teleop visuals for sequential static manipulation task docs

9ef514b

update data gen docs

08032ea

fix teleop command

235a0b2

doc for 4&5

e96e9f1

format

ce1ea35

make html

fac7afe

format

103fb87

xyao-nv force-pushed the peterd/gtc_dli_docs branch from 8d0cae1 to 103fb87 Compare February 6, 2026 18:49

xyao-nv and others added 7 commits February 6, 2026 11:07

fix cmds

855758b

fix path

753f3eb

add mimic subtask annotation note and fix revolute joint metric compu…

ec81df2

…tation for closing door

add mimic subtask annotation note and fix revolute joint metric compu…

ec4ee1e

…tation for closing door

Merge branch 'peterd/gtc_dli_docs' of github.com:isaac-sim/IsaacLab-A…

b929052

…rena into peterd/gtc_dli_docs

fix doc warning

11c491a

disable camera for mimic seq data gen test to prevent flakey test

ed8e087

peterd-NV changed the title ~~Draft: GTC DLI workflow docs~~ GTC DLI workflow docs Feb 6, 2026

peterd-NV merged commit e81d5e4 into main Feb 6, 2026
5 checks passed

Conversation

peterd-NV commented Feb 5, 2026

Summary

Uh oh!

xyao-nv commented Feb 6, 2026

Uh oh!

xyao-nv commented Feb 6, 2026

Uh oh!

alexmillane left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants