add vel and torque to lerobot conversion

[shokubutsuu]

previous implementation missed the vel and torque information in each record, this pr add these into the flow, keeping the data complete for future use.

[kou]

Claude Code says the following. What do you think about it?

---

Is it intentional that observation.velocity and observation.torque are smoothed in the same way as observation.state?

Currently qvel and qtorque go through the existing smoothing path:

1. to_lerobotv21()/to_lerobotv30() call dataset.set_smoothing(cutoff=smoothing_cutoff) (default: 1.0 Hz)
2. Dataset._load_embodiment_values() applies _apply_smoothing() to every loaded attribute, including qvel/qtorque
3. _apply_smoothing() is a 4th order Butterworth low-pass filter with filtfilt

A 1 Hz low-pass filter may be acceptable for positions, but I think it's too strong for torque and velocity. For example, short torque spikes from contacts or collisions are mostly removed. This doesn't match the motivation of this PR: "keeping the data complete for future use".

How about one of the following?

• Don't smooth qvel/qtorque (just resample them to the target FPS)
• Use a separate cutoff for them (e.g. --arm-dynamics-smoothing-cutoff)

If smoothing them with the same cutoff is intentional, could you explain the reason and document it in the README?

[kou]

It seems that we can remove this function.

[kou]

Could you remove default=None because this is a boolean option? (default=False is the default)

Suggested change

default=None,

[shokubutsuu]

This is also one thing that I'm not sure of. I see that previously we don't really turn off smoothing when the value is set to 0. I think it would be better if we keep a version that simply convert the data as it is because once I have them visualized, i see quite a difference between the completely raw one vs a slightly smoothed one. I suggest to set 1 hz manually when there's a need for smoothing the dataset.

[kou]

OK. Let's change the --smoothing-cutoff 0 behavior to disable smoothing cutoff. But let's work on it as a separated PR.

BTW, Claude Code's question was a different one: should qvel/qtorque be smoothed with the same cutoff as qpos by default? With the default --smoothing-cutoff 1.0, short torque/velocity changes are mostly removed. What do you think about it?

[shokubutsuu]

Typically if a model is training on a 1hz dataset, the model usually don't use torque and velocity. Adding a smaller cutoff is doable for the torque and velocity data, but I don't think that would help the model too much to know what's actually happening here, since the gap for videos and actions are too long when smoothing it to 1hz.

I would suggest to put the 0 feature in this pr, since they're actually related to each other. What do you think?

[shokubutsuu]

I'm also changing the --no-arm-dynamics to --arm-dynamics so it doesn't keep these information by default.

[kou]

Just to confirm: --smoothing-cutoff 1.0 doesn't make a 1 Hz dataset. The output is still sampled at --fps (30 by default). It's the cutoff frequency of the low-pass filter. It removes changes faster than about 1 Hz from each stream before resampling. So observation.torque/observation.velocity have 30 values per second but short spikes are mostly removed.

For the --smoothing-cutoff 0 change: I prefer a separated PR for easy to review... Can I open a PR for it?

[shokubutsuu]

sure makes sense! thank you!

[kou]

How about #77 ?

[shokubutsuu]

Thanks for the pr! It looks good!

[kou]

Merged! Could you rebase on main?

[kou]

Can I push some minor cleanups such as #76 (comment) and #76 (comment) to this branch before we merge this?

[shokubutsuu]

i did those changes in my branch so no worries

[kou]

Thanks! Merged!