You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Welcome to the **IROS 2025 Challenge of Multimodal Robot Learning in InternUtopia and Real World**!
8
+
InternManip provides the **official baseline and evaluation toolkit** for **Track: Vision-Language Manipulation in Open Tabletop Environments**, featured at the *[IROS 2025 Workshop](https://internrobotics.shlab.org.cn/workshop/2025/)*.
9
+
10
+
## 🚀 Challenge Overview
11
+
12
+
In this challenge, participants will develop end-to-end policies that fuse vision and language to control robots in simulated physics-based environment.
13
+
Models are trained using the **InternManip** framework and **GenManip** dataset, and evaluated in a closed-loop benchmark on unseen private scenes.
14
+
15
+
This repository serves as the **starter kit and evaluation toolkit**—you can use it to:
16
+
- Implement your own policy models
17
+
- Train them on GenManip public data
18
+
- Submit them via Docker for final evaluation
19
+
20
+
## 📚 More information
21
+
You can get information about the competition [here](https://github.com/InternRobotics/InternManip/blob/master/challenge/README.md), including resources, time and rewards, etc.
22
+
23
+
## 🛠️ guided tutorial
24
+
25
+
We've provided a concise [guided tutorial](https://github.com/InternRobotics/InternManip/blob/master/challenge/guidelines.md) for challengers, divided into three parts: Environment Setup, Local Development & Testing, and Packaging & Submission.
26
+
27
+
28
+
> 😄 Good luck, and we look forward to your innovations!
If you encounter the following error during the genmanip installation:
389
-
```bash
390
-
CondaToNonInteractiveError: Terms of Service have not been accepted for the following channels:
391
-
• https://repo.anaconda.com/pkgs/main
392
-
• https://repo.anaconda.com/pkgs/r
393
-
```
394
-
Manually accept the Terms of Service for each affected channel by running these commands:
395
-
```bash
396
-
conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/main
397
-
conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/r
398
-
```
399
-
400
-
401
-
### 2. Tips for Slow or Unstable Networks
402
-
403
-
If you encounter errors such as timeouts or incomplete downloads, especially in network-restricted or low-bandwidth environments, we recommend the following approaches.
404
-
405
-
- Step 1: By default, `uv pip` uses relatively short HTTP timeouts. To extend the timeout, set the following environment variable before installation:
406
-
```bash
407
-
export UV_HTTP_TIMEOUT=600 # Timeout in seconds (10 minutes)
408
-
```
409
-
410
-
- Step 2: Locate the package that failed to install. When running the `install.sh` script, if a package fails to install, identify the failing line in the output. This usually indicates which package wasn't downloaded properly.
411
-
- Step 3: Dry-run to preview packages and versions. Use uv pip install --dry-run` to preview which packages (and exact versions) are going to be installed. For example:
### 3. `import simpler_env` Fails Due to Missing Vulkan Library
438
-
439
-
If you encounter the following error when trying to import simpler_env:
440
-
```bash
441
-
>>> import simpler_env
442
-
Traceback (most recent call last):
443
-
...
444
-
ImportError: libvulkan.so.1: cannot open shared object file: No such file or directory
445
-
```
446
-
You can resolve this issue by installing the Vulkan runtime library via `apt`:
447
-
```bash
448
-
sudo apt update
449
-
sudo apt install libvulkan1
450
-
sudo ldconfig
451
-
```
452
-
453
-
454
-
455
-
456
-
### 4. Failed to Install `pyzmq` When Building Model Dependency
457
-
458
-
459
-
When installing the `model` environment and encountering the following error:
460
-
```bash
461
-
Resolved 162 packages in 4.80s
462
-
× Failed to build `pyzmq==27.0.1`
463
-
├─▶ The build backend returned an error
464
-
╰─▶ Call to `scikit_build_core.build.build_wheel` failed (exit status: 1)
465
-
```
466
-
A recommended approach is to build from source by running:
467
-
```bash
468
-
./install.sh --model_bfs
469
-
```
470
-
If this fails with GCC-related errors, please refer to the troubleshooting advice below.
471
-
472
-
### 5. GCC Fails to Compile Due to Missing Dependencies
473
-
474
-
When compiling C++ components (e.g., `building ManiSkill2_real2sim` or `pyzmq`), you might encounter errors related to GCC or missing shared libraries. This guide walks you through how to resolve them without root/sudo permissions.
475
-
476
-
- Step 1: Use a modern GCC (recommended ≥ 9.3.0). Older system compilers (e.g., GCC 5.x or 7.x) may not support required C++ standards. It's recommended to switch to GCC 9.3.0 or newer:
> ⚠️ Note: Simply using a newer compiler might not be enough — it may depend on shared libraries that are not available on your system.
482
-
- Step 2: Manually install required libraries. If you encounter errors like: `error while loading shared libraries: libmpc.so.2 (libmpfr.so.1, libgmp.so.3)`. If you do have `sudo` privileges, the easiest way is to install the required libraries system-wide using your system package manager.
483
-
```bash
484
-
sudo apt update
485
-
sudo apt install gcc-9 g++-9
486
-
```
487
-
Or, you need to manually compile and install the following dependencies locally:
- Step 3 (Optional): Fix Missing `.so` Versions. Sometimes you have the correct library version (e.g., `libgmp.so.10`), but GCC expects an older symlink name (e.g., `libgmp.so.3`). You can fix missing library versions with symlinks.
515
-
- Step 4: Export the Library Path. Make sure the compiler can find your locally installed shared libraries:
### 6. Handling `RuntimeError: vk::PhysicalDevice::createDeviceUnique: ErrorExtensionNotPresent` in SimplerEnv Benchmark
520
-
521
-
522
-
This error is a tricky issue caused by a failed GPU driver installation when running the SimplerEnv benchmark. Many users have reported this problem in the [SimplerEnv GitHub issue #68](https://github.com/simpler-env/SimplerEnv/issues/68).
523
-
524
-
You may try the suggested solutions there, but they do not always work reliably. Therefore, we provide an alternative approach if you can use container images.
525
-
526
-
#### Alternative Solution Using Apptainer Container
527
-
- Make sure you have installed `simpler_env`.
528
-
- Install Apptainer on your system if you haven't already.
529
-
- Download a prebuilt container image with SimplerEnv installed from our [Hugging Face repository](https://huggingface.co/InternRobotics/Manishill2/tree/main).
530
-
- Run the evaluation inside the container using a command similar to the following:
Replace path_to_maniskill2.sif with the actual path to the downloaded container image, and path_to_your_python with the Python interpreter you want to use inside the container.
The default evaluation setup adopts a client-server architecture where the policy (model) and the environment run in separate processes. This improves compatibility and modularity for large-scale benchmarks.
If you encounter the following error during the genmanip installation:
6
+
```bash
7
+
CondaToNonInteractiveError: Terms of Service have not been accepted for the following channels:
8
+
• https://repo.anaconda.com/pkgs/main
9
+
• https://repo.anaconda.com/pkgs/r
10
+
```
11
+
Manually accept the Terms of Service for each affected channel by running these commands:
12
+
```bash
13
+
conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/main
14
+
conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/r
15
+
```
16
+
17
+
18
+
## 2. Tips for Slow or Unstable Networks
19
+
20
+
If you encounter errors such as timeouts or incomplete downloads, especially in network-restricted or low-bandwidth environments, we recommend the following approaches.
21
+
22
+
- Step 1: By default, `uv pip` uses relatively short HTTP timeouts. To extend the timeout, set the following environment variable before installation:
23
+
```bash
24
+
export UV_HTTP_TIMEOUT=600 # Timeout in seconds (10 minutes)
25
+
```
26
+
27
+
- Step 2: Locate the package that failed to install. When running the `install.sh` script, if a package fails to install, identify the failing line in the output. This usually indicates which package wasn't downloaded properly.
28
+
- Step 3: Dry-run to preview packages and versions. Use uv pip install --dry-run` to preview which packages (and exact versions) are going to be installed. For example:
## 3. `import simpler_env` Fails Due to Missing Vulkan Library
55
+
56
+
If you encounter the following error when trying to import simpler_env:
57
+
```bash
58
+
>>> import simpler_env
59
+
Traceback (most recent call last):
60
+
...
61
+
ImportError: libvulkan.so.1: cannot open shared object file: No such file or directory
62
+
```
63
+
You can resolve this issue by installing the Vulkan runtime library via `apt`:
64
+
```bash
65
+
sudo apt update
66
+
sudo apt install libvulkan1
67
+
sudo ldconfig
68
+
```
69
+
70
+
71
+
72
+
73
+
## 4. Failed to Install `pyzmq` When Building Model Dependency
74
+
75
+
76
+
When installing the `model` environment and encountering the following error:
77
+
```bash
78
+
Resolved 162 packages in 4.80s
79
+
× Failed to build `pyzmq==27.0.1`
80
+
├─▶ The build backend returned an error
81
+
╰─▶ Call to `scikit_build_core.build.build_wheel` failed (exit status: 1)
82
+
```
83
+
A recommended approach is to build from source by running:
84
+
```bash
85
+
./install.sh --model_bfs
86
+
```
87
+
If this fails with GCC-related errors, please refer to the troubleshooting advice below.
88
+
89
+
## 5. GCC Fails to Compile Due to Missing Dependencies
90
+
91
+
When compiling C++ components (e.g., `building ManiSkill2_real2sim` or `pyzmq`), you might encounter errors related to GCC or missing shared libraries. This guide walks you through how to resolve them without root/sudo permissions.
92
+
93
+
- Step 1: Use a modern GCC (recommended ≥ 9.3.0). Older system compilers (e.g., GCC 5.x or 7.x) may not support required C++ standards. It's recommended to switch to GCC 9.3.0 or newer:
> ⚠️ Note: Simply using a newer compiler might not be enough — it may depend on shared libraries that are not available on your system.
99
+
- Step 2: Manually install required libraries. If you encounter errors like: `error while loading shared libraries: libmpc.so.2 (libmpfr.so.1, libgmp.so.3)`. If you do have `sudo` privileges, the easiest way is to install the required libraries system-wide using your system package manager.
100
+
```bash
101
+
sudo apt update
102
+
sudo apt install gcc-9 g++-9
103
+
```
104
+
Or, you need to manually compile and install the following dependencies locally:
- Step 3 (Optional): Fix Missing `.so` Versions. Sometimes you have the correct library version (e.g., `libgmp.so.10`), but GCC expects an older symlink name (e.g., `libgmp.so.3`). You can fix missing library versions with symlinks.
132
+
- Step 4: Export the Library Path. Make sure the compiler can find your locally installed shared libraries:
## 6. Handling `RuntimeError: vk::PhysicalDevice::createDeviceUnique: ErrorExtensionNotPresent` in SimplerEnv Benchmark
137
+
138
+
139
+
This error is a tricky issue caused by a failed GPU driver installation when running the SimplerEnv benchmark. Many users have reported this problem in the [SimplerEnv GitHub issue #68](https://github.com/simpler-env/SimplerEnv/issues/68).
140
+
141
+
You may try the suggested solutions there, but they do not always work reliably. Therefore, we provide an alternative approach if you can use container images.
142
+
143
+
**Alternative Solution Using Apptainer Container**
144
+
- Make sure you have installed `simpler_env`.
145
+
- Install Apptainer on your system if you haven't already.
146
+
- Download a prebuilt container image with SimplerEnv installed from our [Hugging Face repository](https://huggingface.co/InternRobotics/Manishill2/tree/main).
147
+
- Run the evaluation inside the container using a command similar to the following:
Replace path_to_maniskill2.sif with the actual path to the downloaded container image, and path_to_your_python with the Python interpreter you want to use inside the container.
0 commit comments