Installation and Startup

• Host requirements: Ubuntu 22.04 on x86/arm64 with a compatible ROS environment installed.
• After extracting the project package (ROS required on the host), you can use it in three ways:

  • Desktop shortcut (default UI): Run ./scripts/install-desktop.sh in the project directory. A desktop icon will appear; double-click to open the app in your browser.

    

  • Terminal launch (with UI): Run ./scripts/run_gateway.sh in the project directory.

  • Headless (no UI): ./scripts/run_gateway.sh --headless.

UI Overview

The frontend is organized top to bottom as:

• Exoskeleton and dexterous hand configuration
• Exoskeleton and dexterous hand visualization
• System monitor

Exoskeleton Glove Configuration and Visualization

Connecting the Exoskeleton Glove

Wired connection

• Connect the exoskeleton glove to a USB port on your PC. The software automatically detects left/right hand connection status (refresh takes ~3 seconds).

Wireless connection

First-time wireless module provisioning

warning

Provision each wireless module separately: complete provisioning for one module, power it off, then provision the other. After both modules are provisioned, you can power them on for normal use.

Provisioning flow

1. Power on the wireless module:
   • Short press + long press the device button; release when the battery LED blinks to power on.
2. Enter pairing mode:
   • Step 1: While powered on: short press → long press 3 s (battery LED blinks once) → continue holding to 10 s (blue LED on) → release → device powers off;
   • Step 2: Power on again: short press → long press 3 s (battery LED blinks once) → release → pairing mode (blue LED on).
3. ESP provisioning:
   • Connect the included router to the PC Ethernet port and ensure PC Wi-Fi is on the router's 2.4 GHz band (e.g. IO_2.4G_*****);
   • Enter router Wi-Fi SSID, password, and callback IP (use the router gateway IP from the PC);
   • Click Start provisioning; the software validates the callback IP (click Save configuration to persist network settings locally);
   • Watch the module LED: blue solid → brief red → green solid.
4. Device connection:
   • Connect the wireless module to the exoskeleton device; the LED should blink green;
   • Click Start UDP receiver in the UI and wait for completion;
   • The system auto-detects and binds left/right devices; on success, port info and connected status appear in the UI.

Wireless module button reference

Function	Action
Power on	Short press → long press 3 s (battery LED blinks once) → release
Power off	Short press → long press 3 s (battery LED blinks once) → release
Check battery (powered off)	Short press
Enter pairing mode	1. Powered on: short press → long press 3 s (battery LED blinks once) → continue to 10 s (blue LED on) → release → powers off 2. Power on again: short press → long press 3 s (battery LED blinks once) → release → pairing mode (blue LED on)

Wireless module LED reference

State	LED
No Wi-Fi	Red solid
Listen / pairing mode	Blue solid
Wi-Fi connected, no device data	Green solid
Wi-Fi connected, device data active	Green blinking
Reading intrinsics	Blue blinking
Device discovered	Blue-green blinking

Exoskeleton Glove Visualization

Real-time motion visualization

• Drag with the mouse to rotate; scroll to zoom.

Left/right joint data

• Shows exo_joint values over time for each side.
• Toggle series to display.

Output frequency

• Shows exo_joint output frequency over time.
• Chart axis can be fixed or dynamic.

Vibration feedback

• Shows exo_vibration intensity sent from the dexterous hand to the exoskeleton.
• Endpoints 1–10 map to the 10 exoskeleton fingertips.

Dexterous Hand Configuration and Visualization

Dexterous hand configuration

• Upload hand model configuration:
  • Drag and drop archives (zip/tar.gz/tgz, etc.) or folders;
  • Or click Upload configuration and select in the file browser.
• Select target hand model: click Refresh model list if the model is missing.
• Click Apply model to activate.

Dexterous hand visualization

Real-time hand motion visualization

• Drag with the mouse to rotate; scroll to zoom.

Left/right joint data

• Shows cmd_left / cmd_right values over time.
• Toggle series to display.

Left/right output frequency

• Shows cmd_left / cmd_right output frequency over time.
• Chart axis can be fixed or dynamic.

System Monitor

Status

• Live /status data: runtime topology, hand model, side, subprocesses, ROS bridge, sync state.

WebSocket data

• Live values for exoskeleton exo_joint and applied dexterous hand cmd_left / cmd_right.
• Select streams from the dropdown.

REST API

• Prefix: /api/v1
• Base URL: http://{host}:{port}/api/v1/...
• Content-Type: JSON (except upload endpoints)

Method	Path	Purpose	Example
GET	/streams	WebSocket stream catalog (id, topic, type, direction)	curl -s http://127.0.0.1:8080/api/v1/streams
GET	/status	Runtime status: topology, hand model, side, subprocesses, ROS bridge, sync	curl -s http://127.0.0.1:8080/api/v1/status
GET	/hands/configs	List valid hand models (scans configs/end_tools)	curl -s http://127.0.0.1:8080/api/v1/hands/configs
POST	/hands/select	Select hand model and start process chain (exo/transform/controller)	—
POST	/hands/configs/upload	Upload hand config to configs/end_tools/<hand>/	—
POST	/runtime/frequency	Set exo publish rate 120 (default) / 100 Hz; writes yaml and restarts exo_tf	—
POST	/wifi/provision	Start Wi-Fi provisioning	—
POST	/exo/udp/start	Start UDP exoskeleton discovery	—
GET	/exo/udp/stop	Stop UDP discovery	—

WebSocket Data Streams

Stream catalog

base_id	scope	ROS topic (template)	ROS type	direction
io_esk.tf	global	/io_fusion/tf_exoskeleton	TFMessage	subscribe
io_esk.joint_data	global	/io_esk/joint_data	JointState	subscribe
io_esk.joystick_data	global	/io_esk/joystick_data	Joy	subscribe
io_align.tf.robotname	hand	/io_align/tf_hand	TFMessage	subscribe
io_align.poses_left.robotname	hand	/io_align/<hand>/poses_<new_parent_frame>	PoseArray	subscribe
io_align.poses_right.robotname	hand	/io_align/<hand>/poses_<new_parent_frame>	PoseArray	subscribe
io_teleop.joint_cmd_left.robotname	hand	/io_teleop/joint_cmd_finger_left	JointState	subscribe
io_teleop.joint_cmd_right.robotname	hand	/io_teleop/joint_cmd_finger_right	JointState	subscribe
io_esk.vibration_feedback	—	/io_esk/vibration_feedback	Float64MultiArray	publish

Stream frame format

Timestamps use stamp_ns (int64, nanoseconds).

Field	Type	Description
stream	string	stream id
data	object	ROS message as JSON

{
  "stream": "exo_joint",
  "data": {
    "stamp_ns": 1717401234567890123,
    "names": ["joint1", "joint2", "..."],
    "position": [0.1, 0.2, "..."],
    "velocity": [],
    "effort": []
  }
}

Data types by stream

TFMessage — exo_tf, align_tf.*

{
  "stamp_ns": 1717401234567890123,
  "transforms": [
    {
      "parent": "world",
      "child": "base_link",
      "translation": [0.0, 0.0, 0.0],
      "rotation": [0.0, 0.0, 0.0, 1.0]
    }
  ]
}

JointState — exo_joint, cmd_left.*, cmd_right.*, teleop_joint_states.*

{
  "stamp_ns": 1717401234567890123,
  "names": ["joint1", "joint2", "..."],
  "position": [0.1, 0.2, "..."],
  "velocity": [],
  "effort": []
}

Joy — exo_joy

{
  "stamp_ns": 1717401234567890123,
  "axes": [0.0, 0.0, 0.0, 0.0],
  "buttons": [0, 1, 0, 0, 0, 0, 0, 0]
}

PoseArray — align_poses_left_hand_ee_link.*, align_poses_right_hand_ee_link.*

{
  "stamp_ns": 1717401234567890123,
  "poses": [
    {
      "position": [0.0, 0.0, 0.0],
      "orientation": [0.0, 0.0, 0.0, 1.0]
    }
  ]
}

Float64MultiArray — publish exo_vibration

{"data": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]}