[Bug Report] Camera output resolution stays the same across camera instances

[jnzhnd]

Describe the bug

My current simulation setup is as follows: I have two cameras of different resolutions, and I want one of them to create a RGB point cloud, and the other one to capture a normal image of a scene. This should happen only once for performance reasons, so I create one camera object, take a picture, then delete it, then create the other camera object, create a point cloud and delete it again.

When I do this, the RGB output of the second camera still has the same resolution (i.e., shape of the underlying torch tensor) as the first camera. The "distance_to_image_plane" output (which the first camera obviously does not have) has the correct resolution though.

If I then want to create a point cloud from the data of the second camera, it obviously fails:

The shape of the mask [403680] at index 0 does not match the shape of the indexed tensor [4194304, 3] at index 0

Could it be that some buffers are not properly reset when the camera is deleted? And if so, can I do this manually?

Steps to reproduce

So a (very reduced) minimal working example of what I'm trying to do is as follows:

from omni.isaac.orbit.app import AppLauncher
app_launcher = AppLauncher()
simulation_app = app_launcher.app

from omni.isaac.orbit.sensors.camera import Camera, CameraCfg
import omni.isaac.orbit.sim as sim_utils
from omni.isaac.orbit.sensors import utils as cam_utils
import omni.isaac.core.utils.prims as prim_utils

def main():

    cfg = sim_utils.SimulationCfg(dt=0.01, use_gpu_pipeline=True, device="cuda:0")
    sim = sim_utils.SimulationContext(cfg)
    
    rgb_camera_cfg = CameraCfg(
        prim_path=None,
        update_period=0,
        height=2048,
        width=2048,
        data_types=["rgb"],
        spawn=sim_utils.PinholeCameraCfg(
            focal_length=24.0, focus_distance=400.0, horizontal_aperture=20.955, clipping_range=(0.1, 1.0e5)
            ),
    )
    
    d_camera_cfg = CameraCfg(
        prim_path=None,
        update_period=0,
        height=480,
        width=841,
        data_types=["distance_to_image_plane", "rgb", "semantic_segmentation"],
        spawn=sim_utils.PinholeCameraCfg(
            focal_length=20.0, focus_distance=400.0, horizontal_aperture=20.955, clipping_range=(0.1, 1.0e5)
            ),
    )
    
    sim.reset()

    count = 0
    while simulation_app.is_running():
        if count == 100:
            # Record data for rgb image
            sim.pause()
            camera = Camera(cfg=rgb_camera_cfg.replace(prim_path="/RGB_Camera"))
            sim.play()
            camera.update(dt = 0.01)
            for _ in range(3):
                sim.step()
            rgb_data = camera.data.output[0]
            camera.__del__()
            prim_utils.delete_prim(prim_path="/RGB_Camera")
            prim_utils.delete_prim(prim_path="/Replicator")
            
            # Record data for point cloud
            sim.pause()
            camera = Camera(cfg=d_camera_cfg.replace(prim_path="/D_Camera"))
            sim.play()
            camera.update(dt = 0.01)
            for _ in range(3):
                sim.step()
            d_data = camera.data.output[0]
            d_intrinsics = camera.data.intrinsic_matrices
            d_pos = camera.data.pos_w
            d_orient = camera.data.quat_w_opengl
            camera.__del__()
            prim_utils.delete_prim(prim_path="/D_Camera")
            prim_utils.delete_prim(prim_path="/Replicator")

            # Create point cloud
            cloud = cam_utils.create_pointcloud_from_rgbd(intrinsic_matrix=d_intrinsics,
                                                              depth=d_data["distance_to_image_plane"],
                                                              rgb=d_data["rgb"],
                                                              normalize_rgb=True,
                                                              position=d_pos,
                                                              orientation=d_orient,
                                                              )
        count += 1

If I print the camera.data.output[0] while running this, i get the following for my rgb camera (the one that's supposed to be generating an image:

TensorDict(
    fields={
        rgb: Tensor(shape=torch.Size([2048, 2048, 4]), device=cuda:0, dtype=torch.uint8, is_shared=True)},
    batch_size=torch.Size([]),
    device=cuda:0,
    is_shared=True)

And the following for the depth camera (the one that's supposed to be creating a point cloud):

TensorDict(
    fields={
        distance_to_image_plane: Tensor(shape=torch.Size([480, 841]), device=cuda:0, dtype=torch.float32, is_shared=True),
        rgb: Tensor(shape=torch.Size([2048, 2048, 4]), device=cuda:0, dtype=torch.uint8, is_shared=True),
        semantic_segmentation: Tensor(shape=torch.Size([480, 841]), device=cuda:0, dtype=torch.int32, is_shared=True)},
    batch_size=torch.Size([]),
    device=cuda:0,
    is_shared=True)

System Info

• Using the devel branch of Orbit
• Commit: aaab27b
• Isaac Sim Version: 2023.1.0-hotfix.1
• OS: Ubuntu 22.04
• GPU: RTX 3090
• CUDA: 12.0
• GPU Driver: 525.147.05

Checklist

• I have checked that there is no similar issue in the repo (required)
• I have checked that the issue is not in running Isaac Sim itself and is related to the repo

[pascal-roth]

cannot replicate this issue, added a test that it is checked for automatically