Add support for converting existing datasets not for OpenArm to OpenArm Dataset

[kou]

Describe the feature you want

There are some open datasets such as DROID licensed under CC BY 4.0. They may not be collected for OpenArm but some (most?) of them will be useful for trailing policies for OpenArm.

If we can convert these datasets by openarm-dataset-convert, we may be able to build better policies for OpenArm by these converted datasets.

Let's list target open datasets as the first step.

[trakshan-mishra]

Thanks for opening this — I've been looking into the conversion path and want to
share what I found before proposing code. Per the issue, the first step is
listing candidate external datasets; below is a short list with the licensing
and the on-disk semantics I could verify from each project's own source, then
two design questions where I'd like maintainer direction.

Candidate datasets (license + structure verified from each project's source)

• DROID — CC BY 4.0 (droid-dataset.github.io).
  Franka Panda / FR3 7-DoF. RLDS (tfds.load("droid")) + raw HDF5. Per step:
  observation/{joint_position(7), cartesian_position(6), gripper_position(1)}
  • 3 RGB cameras (wrist + 2 exterior, left stereo, 180×320).
    cartesian_position = [x,y,z] + euler (extrinsic xyz, radians) in the
    Franka base frame; gripper_position is normalized [0,1] with 1=closed;
    the flat action is velocity-like. Raw HDF5 carries real wall-clock
    robot_timestamp_nanos; the RLDS view exposes only a 15 Hz sequence index.
• RH20T — mixed license: CC BY-SA 4.0 (commercial subset, scenes
  0001–0005) and CC BY-NC 4.0 (non-commercial subset, scenes 0006–0010)
  (rh20t.github.io). Heterogeneous (7 robot configs),
  pose stored as xyz+quat (7-D) in base and per-camera frames, plus
  force/torque, tactile, audio. Non-commercial subset can't feed a shipped
  policy, so only the CC BY-SA subset is broadly usable.
• Open X-Embodiment — an umbrella, not a single dataset: RLDS format with
  mixed per-dataset licenses (open_x_embodiment).
  Good as a long-term target, less useful as a first direct importer.
• BridgeData V2 — WidowX arm, RLDS/TFDS
  (rail-berkeley/bridge_data_v2).
  Listed as a candidate; I could not confirm a license from the repo README, so
  it needs checking before any import.
• RoboSet — listed as a candidate; I have not yet verified its license, so
  treat as unconfirmed.

DROID is the strongest first target: permissive license, a single embodiment,
RLDS + raw access, and a documented schema. RH20T is richer but its mixed/
non-commercial license and robot heterogeneity make it a harder first step.

Two blockers I hit while mapping DROID → OpenArm

A. A non-OpenArm embodiment can't be declared honestly.
metadata.py (Embodiments._build_embodiment) only accepts OpenArm and
OpenArmCellLifter and raises ValueError for anything else. So a converted
DROID dataset can't declare its Franka embodiment. Labeling it id: OpenArm
would be false — a Franka 7-vector is not an OpenArm 8-vector (the embodiment,
joint conventions, and gripper all differ), and the kinematics/FK-IK path assumes
OpenArm. (Note: Leader intentionally tolerates unknown ids — it's a label —
but Embodiments does not, which looks deliberate since embodiments drive
kinematics.)

B. Foreign Cartesian pose can't be copied into OpenArm pose.
kinematics.py documents that OpenArm pose is an end-effector pose in the
MuJoCo model's world frame; pose recorded in another frame produces "wrong
but plausible-looking" qpos. DROID's cartesian_position is in the Franka
base frame (euler, not quaternion). The euler→quaternion reorder
(R.from_euler("xyz", e).as_quat() is scipy xyzw; OpenArm stores qw,qx,qy,qz)
is well-defined and testable, but without a defined world←base transform the
resulting pose is still in the wrong frame. I did not copy it.

For the same reason I'm not silently mapping: Franka joint_position→OpenArm
qpos, normalized DROID gripper→OpenArm gripper, DROID velocity actions→OpenArm
absolute actions, or RLDS 15 Hz sequence time→timestamp[ns].

Two questions

Q1 — Embodiment/provenance: For imported data, how should a non-OpenArm
source embodiment be represented? Options I see: a new generic/source
embodiment id; provenance metadata separate from the target embodiment; or
another mechanism you have in mind. This also covers source-dataset provenance
(e.g. "converted from DROID"), which the current metadata schema has no slot for.

Q2 — Foreign pose frame: How should pose from a foreign robot/frame be
represented? Options I see: an explicit source→OpenArm-world transform recorded
in metadata; preserve the source frame without enabling qpos conversion; leave
pose unfilled until frame semantics are representable; or another mechanism.

Once these two are settled I have a clear path for a DROID importer, kept in a
separate module (e.g. src/openarm_dataset/external/droid.py) rather than a
branch in Dataset.__init__, emitting only fields with defensible semantics.
I held off on the importer itself to avoid guessing on the embodiment and frame
decisions.

(Separately — and not part of this issue — I noticed openarm-dataset-validate
has no pose checks: a zero-norm or non-finite quaternion in a pose column
passes validation today and then produces NaN downstream in the rot6d
conversion. I'm preparing that as a small, independent validation PR so it
doesn't get tangled with the importer.)

[k1000dai]

I will check soon !