Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ Table of Contents
:titlesonly:

source/setup/quickstart
source/overview/own-project/index
source/setup/walkthrough/index
source/tutorials/index
source/how-to/index
Expand All @@ -104,8 +105,7 @@ Table of Contents
source/overview/core-concepts/index
source/overview/environments
source/overview/reinforcement-learning/index
source/overview/teleop_imitation
source/overview/augmented_imitation
source/overview/imitation-learning/index
source/overview/showroom
source/overview/simple_agents

Expand Down
2 changes: 1 addition & 1 deletion docs/source/how-to/write_articulation_cfg.rst
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ properties of an :class:`~assets.Articulation` in Isaac Lab.
We will use the Cartpole example to demonstrate how to create an :class:`~assets.ArticulationCfg`.
The Cartpole is a simple robot that consists of a cart with a pole attached to it. The cart
is free to move along a rail, and the pole is free to rotate about the cart. The file for this configuration example is
``source/isaaclab_assets/isaaclab_assets/robots/cartpole.py``.
``source/isaaclab_assets/isaaclab_assets/robots/cartpole.py``.

.. dropdown:: Code for Cartpole configuration
:icon: code
Expand Down
1 change: 0 additions & 1 deletion docs/source/overview/developer-guide/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,3 @@ using VSCode.
VS Code <vs_code>
repo_structure
development
template
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ Example usage for the cube stacking task:
Running Cosmos for Visual Augmentation
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

After converting the demonstrations to MP4 format, you can use a `Cosmos <https://github.com/NVIDIA/Cosmos?tab=readme-ov-file>`_ model to visually augment the videos. Follow the Cosmos documentation for details on the augmentation process. Visual augmentation can include changes to lighting, textures, backgrounds, and other visual elements while preserving the essential task-relevant features.
After converting the demonstrations to MP4 format, you can use a `Cosmos`_ model to visually augment the videos. Follow the Cosmos documentation for details on the augmentation process. Visual augmentation can include changes to lighting, textures, backgrounds, and other visual elements while preserving the essential task-relevant features.

We use the RGB, depth and shaded segmentation videos from the previous step as input to the Cosmos model as seen below:

Expand All @@ -99,7 +99,7 @@ We provide an example augmentation output from `Cosmos Transfer1 <https://github
:align: center
:alt: Cosmos Transfer1 augmentation output

We recommend using the `Cosmos Transfer1 <https://github.com/nvidia-cosmos/cosmos-transfer1/tree/e4055e39ee9c53165e85275bdab84ed20909714a>`_ model for visual augmentation as we found it to produce the best results in the form of a highly diverse dataset with a wide range of visual variations. You can refer to the installation instructions `here <https://github.com/nvidia-cosmos/cosmos-transfer1/blob/e4055e39ee9c53165e85275bdab84ed20909714a/INSTALL.md#environment-setup>`_, the checkpoint download instructions `here <https://github.com/nvidia-cosmos/cosmos-transfer1/blob/e4055e39ee9c53165e85275bdab84ed20909714a/examples/inference_cosmos_transfer1_7b.md#download-checkpoints>`_ and `this example <https://github.com/nvidia-cosmos/cosmos-transfer1/blob/e4055e39ee9c53165e85275bdab84ed20909714a/examples/inference_cosmos_transfer1_7b.md#example-2-multimodal-control>`_ for reference on how to use Transfer1 for this usecase. We further recommend the following settings to be used with the Transfer1 model for this task:
We recommend using the `Cosmos Transfer1 <https://github.com/nvidia-cosmos/cosmos-transfer1/tree/e4055e39ee9c53165e85275bdab84ed20909714a>`_ model for visual augmentation as we found it to produce the best results in the form of a highly diverse dataset with a wide range of visual variations. You can refer to the `installation instructions <https://github.com/nvidia-cosmos/cosmos-transfer1/blob/e4055e39ee9c53165e85275bdab84ed20909714a/INSTALL.md#environment-setup>`_, the `checkpoint download instructions <https://github.com/nvidia-cosmos/cosmos-transfer1/blob/e4055e39ee9c53165e85275bdab84ed20909714a/examples/inference_cosmos_transfer1_7b.md#download-checkpoints>`_ and `this example <https://github.com/nvidia-cosmos/cosmos-transfer1/blob/e4055e39ee9c53165e85275bdab84ed20909714a/examples/inference_cosmos_transfer1_7b.md#example-2-multimodal-control>`_ for reference on how to use Transfer1 for this usecase. We further recommend the following settings to be used with the Transfer1 model for this task:

.. rubric:: Hyperparameters

Expand Down
11 changes: 11 additions & 0 deletions docs/source/overview/imitation-learning/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
Imitation Learning
==================

In this section, we show existing scripts for running imitation learning
with Isaac Lab.

.. toctree::
:maxdepth: 1

augmented_imitation
teleop_imitation
Original file line number Diff line number Diff line change
Expand Up @@ -413,7 +413,7 @@ Annotations denote the end of a subtask. For the pick and place task, this means
Each demo requires a single annotation between the first and second subtask of the right arm. This annotation ("S" button press) should be done when the right robot arm finishes the "idle" subtask and begins to
move towards the target object. An example of a correct annotation is shown below:

.. figure:: ../_static/tasks/manipulation/gr-1_pick_place_annotation.jpg
.. figure:: ../../_static/tasks/manipulation/gr-1_pick_place_annotation.jpg
:width: 100%
:align: center

Expand Down
14 changes: 14 additions & 0 deletions docs/source/overview/own-project/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
.. _own-project:

Build your Own Project or Task
==============================

To get started, first create a new project or task with the template generator :ref:`template-generator`.
For more detailed information on how your project is structured, see :ref:`project-structure`.

.. toctree::
:maxdepth: 1
:titlesonly:

template
project_structure
44 changes: 44 additions & 0 deletions docs/source/overview/own-project/project_structure.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
.. _project-structure:


Project Structure
=================

There are four nested structures you need to be aware of when working in the direct workflow with an Isaac Lab template
project: the **Project**, the **Extension**, the **Modules**, and the **Task**.

.. figure:: ../../_static/setup/walkthrough_project_setup.svg
:align: center
:figwidth: 100%
:alt: The structure of the isaac lab template project.

The **Project** is the root directory of the generated template. It contains the source and scripts directories, as well as
a ``README.md`` file. When we created the template, we named the project *IsaacLabTutorial* and this defined the root directory
of a git repository. If you examine the project root with hidden files visible you will see a number of files defining
the behavior of the project with respect to git. The ``scripts`` directory contains the ``train.py`` and ``play.py`` scripts for the
various RL libraries you chose when generating the template, while the source directory contains the python packages for the project.

The **Extension** is the name of the python package we installed via pip. By default, the template generates a project
with a single extension of the same name. A project can have multiple extensions, and so they are kept in a common ``source``
directory. Traditional python packages are defined by the presence of a ``pyproject.toml`` file that describes the package
metadata, but packages using Isaac Lab must also be Isaac Sim extensions and so require a ``config`` directory and an accompanying
``extension.toml`` file that describes the metadata needed by the Isaac Sim extension manager. Finally, because the template
is intended to be installed via pip, it needs a ``setup.py`` file to complete the setup procedure using the ``extension.toml``
config. A project can have multiple extensions, as evidenced by the Isaac Lab repository itself!

The **Modules** are what actually gets loaded by Isaac Lab to run training (the meat of the code). By default, the template
generates an extension with a single module that is named the same as the project. The structure of the various sub-modules
in the extension is what determines the ``entry_point`` for an environment in Isaac Lab. This is why our template project needed
to be installed before we could call ``train.py``: the path to the necessary components to run the task needed to be exposed
to python for Isaac Lab to find them.

Finally, the **Task** is the heart of the direct workflow. By default, the template generates a single task with the same name
as the project. The environment and configuration files are stored here, as well as placeholder, RL library dependent ``agents``.
Critically, note the contents of the ``__init__.py``! Specifically, the ``gym.register`` function needs to be called at least once
before an environment and task can be used with the Isaac Lab ``train.py`` and ``play.py`` scripts.
This function should be included in one of the module ``__init__.py`` files so it is called at installation. The path to
this init file is what defines the entry point for the task!

For the template, ``gym.register`` is called within ``isaac_lab_tutorial/source/isaac_lab_tutorial/isaac_lab_tutorial/tasks/direct/isaac_lab_tutorial/__init__.py``.
The repeated name is a consequence of needing default names for the template, but now we can see the structure of the project.
**Project**/source/**Extension**/**Module**/tasks/direct/**Task**/__init__.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
.. _template-generator:

Build your Own Project or Task
==============================

Create new project or task
==========================

Traditionally, building new projects that utilize Isaac Lab's features required creating your own
extensions within the Isaac Lab repository. However, this approach can obscure project visibility and
Expand Down
2 changes: 1 addition & 1 deletion docs/source/setup/walkthrough/concepts_env_design.rst
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ reference frame is what defines the world.

"Above" the world in structure is the **Sim**\ ulation and the **App**\ lication. The **Application** is "the thing responsible for
everything else": It governs all resource management as well as launching and destroying the simulation when we are done with it.
When we :ref:`launched training with the template<walkthrough_project_setup>`, the window that appears with the viewport of cartpoles
When we :ref:`launched training with the template<template-generator>`, the window that appears with the viewport of cartpoles
training is the Application window. The application is not defined by the GUI however, and even when running in headless mode all
simulations have an application that governs them.

Expand Down
1 change: 0 additions & 1 deletion docs/source/setup/walkthrough/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ represents a different stage of modifying the default template project to achiev
:maxdepth: 1
:titlesonly:

project_setup
concepts_env_design
api_env_design
technical_env_design
Expand Down
111 changes: 0 additions & 111 deletions docs/source/setup/walkthrough/project_setup.rst

This file was deleted.

6 changes: 3 additions & 3 deletions source/isaaclab/docs/CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ Added


0.44.12 (2025-08-12)
~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~

Fixed
^^^^^
Expand All @@ -135,7 +135,7 @@ Fixed


0.44.11 (2025-08-11)
~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~

Fixed
^^^^^
Expand All @@ -144,7 +144,7 @@ Fixed


0.44.10 (2025-08-06)
~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~

Fixed
^^^^^
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -393,6 +393,7 @@ def set_external_force_and_torque(
all the external wrenches will be applied in the frame specified by the last call.

.. code-block:: python

# example of setting external wrench in the global frame
asset.set_external_force_and_torque(forces=torch.ones(1, 1, 3), env_ids=[0], is_global=True)
# example of setting external wrench in the link frame
Expand Down
2 changes: 1 addition & 1 deletion tools/template/templates/external/.vscode/tasks.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
"inputs": [
{
"id": "isaac_path",
"description": "Absolute path to the current Isaac Sim installation. Can be skipped if Isaac Sim installed from pip.",
"description": "Absolute path to the current Isaac Sim installation. If you installed IsaacSim from pip, the import of it failed. Please make sure you run the task with the correct python environment. As fallback, you can directly execute the python script by running: ``python.sh <path-to-your-project>/.vscode/tools/setup_vscode.py``",
{% if platform == "win32" %}
"default": "C:/isaacsim",
{% else %}
Expand Down
Loading