Skip to main content

VR App Usage Guide

PICO System Settings​

Safety Boundary Setup​

Go to System Settings > Safety Protection > Set Safety Boundary, click Go to Settings, then follow the video instructions.

Wi-Fi Connection​

In the VR system, click the Wi-Fi icon in the bottom right corner, go to Network Settings, and connect to the TeleSuit Wi-Fi using the password minnanoIO2023.

Wireless Screen Mirroring​

Developer Mode​

Go to Settings > About, and tap the Software Version repeatedly (about 20 times) until the Developer Options prompt appears.

Performance Mode​

Go to Developer Options > Performance Debug Tools, agree to the Risk Notice, select TeleXperience, and optimize image quality with the following settings:

  • Set refresh rate to 90Hz.
  • Set resolution to 1.5x or 2x.
  • Set CPU/GPU and fixed foveated rendering level to High.

This configuration significantly improves image quality but increases power consumption.

Note: Settings need to be reconfigured after app reinstallation. Enterprise PICO requires system version 5.11.3.U or higher.

Interface Overview​

Startup Scene​

Startup animation and safety notice: After entering the scene, the system displays a safety notice with "Agreed" checked by default. After a 3-second countdown, press the A button on the right controller to agree and enter the main control scene. If you uncheck "Agreed", pressing A will exit the application.

Main Control Scene​

main_view

Panel Introduction​

Docker Navigation Bar​

docker导航栏

  • Camera: Camera settings panel toggle
  • MainControl: Main system panel toggle
  • Setting: System settings panel toggle
  • Platform: Teleoperation platform panel toggle

Note: Camera, Setting, and Platform windows are mutually exclusive and share the same layer, so they cannot be displayed simultaneously.

Main Control Center Panel​

The main control center panel contains four sub-panels: TeleXperience, Dashboard, Comm.Frequency, and Battery Level.

TeleXperience Panel​

Used for system control.

TeleXperience

Error panel expansion: In normal state, click the ▽ icon to expand the panel and display system error information.

TeleXperience_ex

Dashboard Panel​

Displays teleoperation-related function status.

dashboard

Comm.Frequency Panel​

Displays network connection status with TeleSuit and TeleBox, and shows the communication frequency of detected messages.

TeleXperience

Battery Level Panel​

Displays battery information for all system modules.

battery_level

Sub-panel Status:

  • Normal state: Normal display state with sub-panel content shown normally.
  • Closed state: After clicking the MainControl icon, the sub-panel closes and no longer displays.
  • Hand-following state: Sub-panel is only visible when raising your hand, appearing or disappearing with hand movements.
  • Hidden state (in hand-following mode): In hand-following state, after clicking the menu button, the sub-panel is hidden and no longer displays.

Sub-panel Status Toggle: Click the ◁ button to switch between normal state and hand-following state.

Hand Menu​
Trigger Method​
  • Raise your left hand so the finger trigger faces your face. A blue button appears in the bottom left corner as the hand menu toggle.
  • If entering the app for the first time, it defaults to hidden state (in hand-following mode). First click the Menu button to enter hand-following state.
  • Keep the controller at a 30° to 45° angle with your line of sight, and the panel in hand state will appear.
Mocap Settings Panel​

Click the mocap setting icon to open/close the motion capture settings panel.

mocap_setting mocap_setting mocap_setting

Trajectory Playback Panel​

Click the play traj icon to open/close the trajectory playback panel.

play_traj

Robot Panel​

The robot panel automatically displays when the system communicates normally with TeleBox.

robot

Mocap Avatar Panel​

The mocap avatar panel automatically displays when Mocap is successfully enabled.

Camera Panel​

camera_view

Platform Panel​

platform

Interaction Methods​

Panel Show/Hide​

Click an icon to show a panel, click the same icon again to hide it.

main_view

Panel Dragging​

Select the panel's valid area with the middle finger trigger, then drag.

  • Applicable Areas

main_view

  • Action Demo

main_view

Panel Distance Adjustment​

Select the panel's valid area with the middle finger trigger, then push/pull the joystick forward/backward to control panel distance.

  • Applicable Areas

main_view

  • Action Demo

main_view

Panel Orientation Adjustment​

Select the panel's valid area with the middle finger trigger, then push the joystick left/right to control panel orientation.

  • Applicable Areas

main_view

  • Action Demo

main_view

Panel Reset​

Long press the corresponding icon to reset the current panel position and size to default.

  • Camera panel reset: Long press the Camera icon

main_view

  • Main system panel reset: Long press the MainControl icon.

main_view

  • Robot panel reset: Long press the 🔄 icon in the robot panel.

main_view

  • Mocap avatar panel reset: Long press the 🔄 icon in the mocap avatar panel.

main_view

Panel Scaling​

When moving the ray to the four corners of a panel window, white sphere anchor points appear. Hold the middle finger trigger and drag the anchor points to scale.

Note: Currently only the camera panel supports this operation.

main_view

Motion Capture Features​

Motion Capture Device Selection​

Click the mocap setting button to open the motion capture settings panel, then click the Device List button on the left. The right list will display currently available devices.

mocap_setting

  • Note: This feature is only available when motion capture is not enabled. If motion capture is already started, device selection is unavailable.

  • You can freely select from supported device combinations. If the selected combination is invalid, the system will automatically revert to the last correct configuration.

  • Supported device combinations:

    • UPPER_BODY
    • UPPER_BODY + LOWER_BODY
    • UPPER_BODY + LOWER_BODY + GLOVE
    • UPPER_BODY + LOWER_BODY + EXOSKELETON_GLOVE
    • UPPER_BODY + GLOVE
    • UPPER_BODY + EXOSKELETON_GLOVE
    • GLOVE
    • EXOSKELETON_GLOVE
    • EXOSKELETON_GLOVE + WIRELESS_MODULE

Motion Capture Calibration​

In the motion capture settings panel, click the Pose Calib or VR Calib button on the left.

mocap_setting mocap_setting

  • Note: This feature is only available when motion capture is enabled.

Motion Capture Pose Calibration​

Refer to the following video to learn the calibration process and key movements.

Click the BODYCalib button, and the system will automatically start the calibration process based on the selected combination.

VR Motion Capture Joint Calibration - VRHead​

Refer to the following video to learn the calibration process and key movements.

Click the CalibVRHeader button, and the system will automatically start the calibration process.

VR Motion Capture Joint Calibration - VRController​

Refer to the following video to learn the calibration process and key movements.

Click the CalibVRController button, and the system will automatically start the calibration process.

Motion Capture Calibration Guide Animation​

  • Enabled by default. After checking Guided Animation, the system will play corresponding animations at each calibration stage as guidance. Refer to the tutorial videos for specific movement details.

Exoskeleton Glove Visualization​

When exoskeleton gloves are enabled and motion capture is active, double-click the upper body of the mocap avatar with the controller index finger trigger to toggle exoskeleton glove visualization.

  • Note: When exoskeleton glove visualization is enabled, closing motion capture will also close visualization. To use it again, double-click the mocap avatar's upper body to enable exoskeleton glove visualization.

gloves_visualization

gloves_visualization_control

Control Features​

End-Effector Position Mapping Scale Adjustment​

Disabled by default. This feature allows adjusting the robot's linear velocity when following the operator's hand movements by adjusting the end-effector position mapping scale factor. The system currently provides one mode: Custom, which can be enabled in the system settings panel.

  • Custom Mode: The scale factor is a user-defined fixed value. After entering the desired coefficient, click the Apply button to apply.
  • Before starting teleoperation, set the scale factor in advance:
    • When coefficient > 1, robot movement speed is greater than hand movement.
    • When coefficient < 1, robot movement speed is less than hand movement.

system_setting_2

Camera Features​

Based on the camera configuration in TeleBox's Dashboard, the camera settings panel displays the corresponding number of camera windows. Each window has a checkbox at the bottom. Check it to display the corresponding camera panel.

main_view

Component Layout Features​

  • In the system settings panel, click Layout, then click Add New to save the current panel position as a layout. Up to 5 layouts can be saved.
  • Select a saved layout template from the dropdown list. Click Delete to remove the current template, or click Apply to apply the selected layout.
  • Click the Reset button to restore all component layouts to default with one click.

Note: If the number of cameras in the current template does not match the actual number of cameras, the layout application will not be executed. Users need to select a template with a matching number.

Status Monitoring Features​

Battery Level Monitoring​

  • VR Headset: Displays headset battery level.
  • L-Con: Displays left controller connection status and remaining battery.
  • R-Con: Displays right controller connection status and remaining battery.
  • TeleSuit: Displays TeleSuit battery level or DC power supply status.

Communication Monitoring​

TeleXperience

  • TeleSuit: Displays connection status with TeleSuit.
  • TeleBox: Displays connection status with TeleBox.
  • JointState: Communication frequency of received robot status information.
  • Video_0: Communication frequency of received camera data.
  • Mocap: Communication frequency of received Mocap data.
  • VrData: Communication frequency of sent VrData.

Note: Click video_0 to switch to displaying communication frequencies for multiple cameras simultaneously.

Function Status Display​

dashboard

  • Sync: Displays teleoperation synchronization status.
  • Record: Displays teleoperation recording status.
  • Count: Displays the number of recorded data entries in the system.
  • Duration: Displays the recorded duration of current recording data.

Network Disconnection Alert​

  • During system operation, the system automatically monitors communication connection status with TeleSuit (10.42.0.2).
  • If the network connection with Suit is interrupted, the system will pop up a network connection failure panel. This panel will remain displayed until the network is restored.

network_warning

Data Playback​

Click the Play Traj button to open the playback panel. Select the data package to play from the dropdown menu, click Confirm to start playback, then click the Play Traj button again to exit.

play_traj

Passthrough Mode​

  • When enabled, the system stops rendering virtual content and only displays real-world camera footage.
  • Quick start/exit: Press both joystick buttons simultaneously to start passthrough mode, press again to exit.

transparency_mode_switch

Safety Features​

Controller Abnormal Pose Filtering​

Enabled by default.

To improve system safety, this feature filters out abnormal large jumps in VR controller pose recognition results. Users can choose whether to enable this feature based on the robot's actual motion performance and set filtering threshold parameters to adapt to different scenarios. After entering the desired threshold parameters, click the Apply button to apply.

system_setting_3

Controller Vibration Feedback​

When vibration data is received, the controller will vibrate. The incoming data mainly controls vibration amplitude, with the following parameters:

  • Vibration amplitude: Received data value multiplied by 0.8 as the actual vibration amplitude
  • Vibration duration: 500 milliseconds
  • Vibration frequency: 100 Hz

Fatigue Detection​

When system runtime reaches a specified threshold (20 minutes, 40 minutes, or 60 minutes), the system automatically pops up a fatigue detection panel to remind users to check their status and rest appropriately.

tired_warnning

Low Battery Detection​

Enabled by default.

  • During program operation, if VR or TeleSuit battery drops below 20%, the system triggers a low battery warning once.
  • If either device's battery drops below 5%, the system automatically disables synchronization after starting sync.
  • To disable this feature, uncheck the Low Power Warning option in the PoseCalib page of the system settings panel.

low_power_warnning

Other Features​

Immersive Mode​

When enabled, the camera panel automatically follows head movements. You can set whether to display the robot and mocap avatar in immersive mode.

immersive_mode

System Version Display​

Enabled by default.

Automatically displays the following information in the upper right corner of the scene:

  • VR version
  • Suit version
  • Box version

Unread Error Notification​

  • When the MainControl panel is closed, if there are errors, the number of unread messages will be displayed in the top right corner of the MainControl icon in the docker bar.
  • The unread count will be automatically cleared after opening the status panel.

Note: Unread notifications are only effective when the panel is in closed state, excluding cases where the panel is hidden or minimized.

Config Switch Monitoring​

After detecting a reapply configuration (Config) operation, the system will perform the following steps:

  1. Re-pull the configuration file.
  2. Restart WebSocket communication.
  3. Reload robot model and camera panels.

Language Switch​

The system currently supports Chinese and English. The default language is English. Users can switch languages in the Language page of the system settings panel.