Request to Release URDF/Mesh and Frame Definition for the Custom UMI Gripper

[SII-dannyXSC]

Hi, thanks for the great work!

The paper describes a custom fingertip UMI device / hand-held UMI gripper used for collecting Hy-UMI demonstrations. Since the gripper geometry and coordinate frame convention are essential for interpreting the released trajectories and reproducing the action/state representation, would it be possible to release:

• the URDF or mesh/CAD files of the UMI gripper;
• the coordinate frame definition of the gripper / end-effector?

This would be very helpful for adapting the released dataset/model to new robot platforms, especially for Track-B style UMI-only cross-embodiment transfer.

Thanks!

[DieofAo]

Hi SII-dannyXSC,
Please find the URDF and mesh files of our custom fingertip UMI gripper in the attachment. The URDF file contains the complete coordinate frame definition of the gripper/end-effector, so you should be able to directly reference it when adapting the dataset/model to new robot platforms.
Please check if the coordinate system definition of urdf matches the dataset. If there are any issues, please point them out.

umi_description.zip

[SII-dannyXSC]

Thanks again for the great work!

I did some additional visualization experiments using a render-based approach with CalibAll. In short, given the UMI pose relative to the camera and a URDF/mesh of the UMI gripper, I rendered the UMI gripper back onto the original video frame and compared the rendered mask with the raw observation.

However, the current overlay does not align very well with the original video. In principle, if the action pose, camera calibration, URDF geometry, and frame convention are all consistent, the rendered mask from the recorded UMI action should closely match the gripper mask in the original video.

Could you provide the exact origin and axis directions of the local coordinate frame for each UMI gripper? For example, where is the gripper-frame origin located, and what physical directions do the local x/y/z axes point to?

Also, could you provide the camera intrinsic parameters used for the released UMI videos, such as fx, fy, cx, cy, image resolution, and whether any distortion correction or undistortion was applied?

This information would be very helpful for correctly rendering the released UMI actions and checking the action-video alignment.

Thanks!

anno_video_h264.9.mp4