SimAiHub Humanoid Robot G1+Inspire Bimanual Cooperative Manipulation Dataset

Data Product Features

• Hardware Configuration: Unitree G1 humanoid robot + Inspire dexterous hands. Bimanual tasks use two dexterous hands cooperatively.
• Manipulation Task Types: Each data episode covers one bimanual cooperative manipulation type:
  • Bimanual Carry: Both hands bear the load simultaneously; the relative pose is approximately rigid during the carry phase. Primary task; the complete decision logic is specified in this design.
  • Stabilize-Operate: One hand maintains contact (bracing a carton, holding a bottle) while the other twists, pulls, or retrieves. Interface. Subtypes: brace_carton、open_bottle、open_drawer.
  • Handover: The object transfers from one hand to the other, with a dual-contact window in between. Interface. Requires recording handover_frame.
  • Assembly: Peg-in-hole or stacking, with tighter tolerance than carrying. Interface. Success is judged by relative pose tolerance.
• Recording: Primarily records wrist poses and dexterous-hand states, including wrist Cartesian target poses, wrist poses, dexterous finger joint commands/states, etc.
• Rewards: Decomposed for offline reinforcement learning: terms for approach, lifting, target tracking, task success, motion smoothness, object drop, and other reward components.
• Labels: 12-stage expert finite-state-machine tags (Perception, Base Alignment, Pre-Grasp, Approach, Closure, Locking, Lifting, Transport, Placement Alignment, Lowering, Release, Retraction), imitation-learning segmentation markers, and success / failure / truncation status flags.
• Domain Randomization: Object XY-plane jitter; 10–20% of episodes are initialized with small tilt angles. Friction coefficients are randomized independently per manipulation type. Data array structures are identical across successful and failed sample sets.
• Sensor Noise Randomization: Baseline configuration adopts reference sensor noise values for G1 and Inspire dexterous hand from official Isaac Lab examples.
• Synchronization and Storage: Each data episode contains clock-aligned observations, rewards, and termination flags synchronized via env.step () in Isaac Lab. Standard HDF5 storage frame rate is 50 Hz.
• Data Source: Each episode data is collected from real-world simulation runs, not generated from existing data.
• Sampling and Timing: HDF5: 50 Hz; MP4 encoding: 10 fps.

Distribution

• Format: HDF5 + MP4.
• Structure: Each episode contains clock-aligned observations, rewards, and termination flags. Each episode covers only one manipulation task type. Successful and failed sample sets have identical array structures.
• Data Volume: Includes normal operation, failure, and recovery data.
• Video: Each episode is saved as MP4. With vision_hz = 10.0 and vision_stride = 5.

Usage

• Application: Training robot bimanual cooperative manipulation policies.
• Application: Sim-to-real transfer research.
• Application: Offline reinforcement learning and imitation learning.

Coverage

• Geographic Coverage: Globally usable.
• Time Range: The original specification does not provide a specific start date or end date.
• Demographics (if applicable): Not applicable.

Custom Development

• Data Scope: This dataset includes 4 bimanual cooperative manipulation types. You may select a subset; pricing will be negotiated separately.
• Partial Customization: Add new bimanual manipulation strategies, adjust the proportion of approach poses, define custom placement targets (surface placement, precision stacking, bin dropping), modify mass and friction-coefficient ranges, adjust total episode count, filter by task-success thresholds, add cameras, repackage data, private delivery.
• Full Customization: Swap robot and dexterous-hand models, re-architect manipulation task logic, rebuild simulation scenarios end-to-end and recollect trajectory data. Domain randomization and sensor noise randomization can be configured according to real-world operational scenarios and hardware-specific sensor noise specifications.
• Future Extension: A long-horizon table tidy task will be added later, involving multiple objects and multiple sub-stages.
• Notice: Default hardware configuration: Unitree G1 + Inspire dexterous hand; single-hand and bimanual actions and custom simulation scenarios can be customized; these changes fall under enterprise custom projects.

License

AI Training Rights

• Use the Data Product to train, fine-tune, and evaluate machine learning models, including large language models.
• Incorporate Data Product content into models and commercialize resulting model outputs.
• Create derivative works (model weights, embeddings, etc.) for any lawful purpose.

• The Data Product itself may not be sold, redistributed, or shared outside of licensed usage.
• Licensee must comply with all applicable laws, including data protection and privacy regulations.

Who Can Use It

• Data Scientists: For training machine learning models.
• Researchers: For academic or scientific studies.
• Businesses: For analysis, insights, or AI development.
• Robotics Engineers: For policy development, sim-to-real transfer, and manipulation benchmarking.

Data Dictionary

| Column Name / Key | Data Type | Description | Possible Values/Notes |
|-------------|-----------|-------------|-----------------------|
| episode_id | string/integer | Episode identifier | Format not specified in the original description |
| observations.wrist_pose | float array | Wrist pose | Single/dual wrist |
| observations.wrist_cartesian_target_pose | float array | Wrist Cartesian target pose | Expert-generated |
| observations.dex_hand_joint_commands | float array | Dexterous finger joint commands | Single/dual hand |
| observations.dex_hand_joint_states | float array | Dexterous finger joint states | Single/dual hand |
| observations.dex_hand_contact_forces | float array | Dexterous-hand contact force data | Single/dual hand |
| observations.vision_step_index | integer array | Proprio step index corresponding to each stored image frame | Used for vision alignment |
| rewards.* | float | Reward components | Approach, lifting, target tracking, task success, motion smoothness, object drop, etc. |
| labels.fsm_stage | categorical/integer | 12-stage expert finite-state-machine tags | Perception, Base Alignment, Pre-Grasp, Approach, Closure, Locking, Lifting, Transport, Placement Alignment, Lowering, Release, Retraction |
| labels.imitation_segmentation | categorical/marker | Imitation-learning segmentation markers | - |
| labels.status_flags | categorical | Success/failure/truncation status flags | success/failure/truncation |
| domain_randomization.* | float/config | Domain randomization settings | XY jitter, tilt angles, friction coefficients |
| sensor_noise.* | config | Sensor noise randomization | Baseline uses official Isaac Lab G1 and Inspire dexterous-hand reference noise values |
| timing.physics_dt | float | Physics time step | 0.005 s |
| timing.decimation | integer | Physics substeps per env step | 4 |
| timing.physics_hz | integer | Physics simulation frequency | 200 Hz |
| timing.env_step_hz | integer | Environment step frequency | 50 Hz |
| timing.hdf5_sample_hz | integer | HDF5 storage sample rate | 50 Hz |
| timing.vision_hz | float | Vision capture rate in simulation time | 10.0 Hz |
| timing.vision_stride | integer | Number of env steps between saved image frames | 5 |
| timing.vision_align | string | Vision-to-proprio alignment rule | obs/rgb_*[i] matches proprio step obs/vision_step_index[i]; train: rgb[t // vision_stride] |
| video_encoding_fps | integer | Recommended MP4 encoding frame rate | 10 fps |