-
Notifications
You must be signed in to change notification settings - Fork 23
GTC DLI workflow docs #390
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
18 commits
Select commit
Hold shift + click to select a range
0e1725e
add docs for gtc dli workflow
peterd-NV b4d256a
add data gen docs for dli course workflow
peterd-NV 60c9b01
clean up step 1 env setup docs
peterd-NV 98a523d
add env visuals
peterd-NV 9ef514b
Add teleop visuals for sequential static manipulation task docs
peterd-NV 08032ea
update data gen docs
peterd-NV 235a0b2
fix teleop command
peterd-NV e96e9f1
doc for 4&5
xyao-nv ce1ea35
format
xyao-nv fac7afe
make html
xyao-nv 103fb87
format
xyao-nv 855758b
fix cmds
xyao-nv 753f3eb
fix path
xyao-nv ec81df2
add mimic subtask annotation note and fix revolute joint metric compu…
peterd-NV ec4ee1e
add mimic subtask annotation note and fix revolute joint metric compu…
peterd-NV b929052
Merge branch 'peterd/gtc_dli_docs' of github.com:isaac-sim/IsaacLab-A…
peterd-NV 11c491a
fix doc warning
peterd-NV ed8e087
disable camera for mimic seq data gen test to prevent flakey test
peterd-NV File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions
3
docs/images/gr1_sequential_static_manipulation_env_vr_view.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
93 changes: 93 additions & 0 deletions
93
docs/pages/example_workflows/sequential_static_manipulation/index.rst
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| GR1 Sequential Pick & Place and Close Door Task | ||
| =============================================== | ||
|
|
||
| This example demonstrates the complete workflow for the **GR1 sequential manipulation task of picking up an object, | ||
| placing it into a refrigerator, and closing the door** in Isaac Lab - Arena, | ||
| covering environment setup and validation, teleoperation data collection, data generation with | ||
| Isaac Lab Mimic, policy post-training, and closed-loop evaluation. | ||
|
|
||
| .. image:: ../../../images/gr1_sequential_static_manipulation_env.gif | ||
| :align: center | ||
| :height: 400px | ||
|
|
||
| Task Overview | ||
| ------------- | ||
|
|
||
| **Task ID:** ``put_item_in_fridge_and_close_door`` | ||
|
|
||
| **Task Description:** The GR1T2 humanoid uses its upper body (arms and hands) to pick up an object, place it into a refrigerator, and close the door. | ||
|
|
||
| **Key Specifications:** | ||
|
|
||
| .. list-table:: | ||
| :widths: 30 70 | ||
| :header-rows: 1 | ||
|
|
||
| * - Property | ||
| - Value | ||
| * - **Tags** | ||
| - Table-top manipulation | ||
| * - **Skills** | ||
| - Pick & place, Close door | ||
| * - **Embodiment** | ||
| - Fourier GR1T2 (54 DOF humanoid) | ||
| * - **Interop** | ||
| - Isaac Lab Teleop, Isaac Lab Mimic | ||
| * - **Scene** | ||
| - LightWheel Kitchen environment | ||
| * - **Objects** | ||
| - Refrigerator (articulated object), Rigid object to be placed in the refrigerator (e.g. ranch dressing bottle) | ||
| * - **Policy** | ||
| - GR00T N1.6 (vision-language foundation model) | ||
| * - **Post-training** | ||
| - Imitation Learning | ||
| * - **Physics** | ||
| - PhysX (200Hz @ 4 decimation) | ||
| * - **Closed-loop** | ||
| - Yes (50Hz control) | ||
|
|
||
|
|
||
| Workflow | ||
| -------- | ||
|
|
||
| This tutorial covers the pipeline pn creating an environment, generating training data, | ||
| fine-tuning a policy (GR00T N1.6), and evaluating the policy in closed-loop. | ||
|
|
||
| Prerequisites | ||
| ^^^^^^^^^^^^^ | ||
|
|
||
| Start the isaaclab docker container | ||
|
|
||
| :docker_run_default: | ||
|
|
||
| You'll also need to create the folders for the data and models. | ||
| Create the folders for the data and models with: | ||
|
|
||
| .. code:: bash | ||
|
|
||
| export DATASET_DIR=/datasets/isaaclab_arena/sequential_static_manipulation_tutorial | ||
| mkdir -p $DATASET_DIR | ||
| export MODELS_DIR=/models/isaaclab_arena/sequential_static_manipulation_tutorial | ||
| mkdir -p $MODELS_DIR | ||
|
|
||
| Workflow Steps | ||
| ^^^^^^^^^^^^^^ | ||
|
|
||
| Follow the following steps to complete the workflow: | ||
|
|
||
| - :doc:`step_1_environment_setup` | ||
| - :doc:`step_2_teleoperation` | ||
| - :doc:`step_3_data_generation` | ||
| - :doc:`step_4_policy_training` | ||
| - :doc:`step_5_evaluation` | ||
|
|
||
|
|
||
| .. toctree:: | ||
| :maxdepth: 1 | ||
| :hidden: | ||
|
|
||
| step_1_environment_setup | ||
| step_2_teleoperation | ||
| step_3_data_generation | ||
| step_4_policy_training | ||
| step_5_evaluation |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🥳