Clarification needed: Is joystick Y-axis inverted? (Dataset description vs inference_viz.py)

[young-cloud-creator]

Hello,

I am a bit confused about the sign convention of the joystick Y-axis and would appreciate some clarification.

In the Hugging Face dataset description( https://huggingface.co/datasets/nvidia/NitroGen ), it states:

Joystick columns are j_left and j_right. They contain x,y coordinates in [-1, 1]. Note that (-1,-1) is the top-left as is standard for joystick axes.

This suggests that when the joystick Y value is negative, it corresponds to pushing the joystick upward.

However, in nitrogen/inference_viz.py (lines 123–124), I noticed the following code and comment:

joy_x = int(mid_x + px * size // 2)
joy_y = int(mid_y - py * size // 2)  # Y is inverted in image coordinates

Based on the generated debug visualization, when py (joystick Y) is negative, the rendered joystick appears to be pushed downward in the image.

This seems inconsistent with the dataset description. Which convention should users rely on when interpreting or using the joystick Y values?

Thank you for your help.

[msln1995]

You are absolutely correct. Good catch!

Upon digging into the codebase, we found that the entire pipeline was actually running on a "private convention" that contradicted the documentation:

Authorization: The training script (train_vlm.py) was generating positive values (+1) for upward movements.
Visualization: The visualization code (inference_viz.py) used subtraction (mid_y - py) to compensate for this positive input, allowing it to render correctly visually despite the wrong sign.
Inference: The controller (fast_controller.py) also expected positive values for upward movement.
So while the code was internally self-consistent (it worked if you ran it as-is), it was indeed inverted relative to the standard joystick protocol and the dataset description (where Up should be -1).