VR Omni-Directional Treadmill
Embedded systems and Unreal Engine integration for a VR treadmill that tracks body movement via Azure Kinect and dynamically adjusts the walking surface in real-time.
The Challenge
A US-based gaming startup was building an omni-directional treadmill for immersive VR locomotion. The system needed to track the user's full-body movement, feed it into a VR game environment, and simultaneously control physical motors to keep the user centered on the treadmill — all in real-time with minimal latency.
Our Approach
We engineered the complete software stack bridging physical hardware with the virtual environment using ROS2 and Unreal Engine.
Body Tracking
An Azure Kinect sensor captures full-body skeletal tracking data. This data is published to ROS2 FastDDS topics, enabling high-speed distribution to all system components simultaneously.
Motor Control
VESC and stepper motor controllers subscribe to the body tracking data via FastDDS. Real-time calculations determine tread direction and rotation speed, dynamically repositioning the user to the treadmill center as they walk in any direction.
VR Integration
The same body tracking data is mapped onto a skeletal mesh in Unreal Engine via OpenXR, creating a 1:1 representation of the user's movement in the virtual world. The user walks naturally while exploring virtual environments.
Technical Stack
- Communication: ROS2 FastDDS
- Tracking: Azure Kinect SDK
- VR Engine: Unreal Engine, OpenXR
- Motor Control: Arduino, VESC controllers
- Languages: C++
- Testing: GTest
Results
- Real-time body tracking to motor control with minimal latency
- Seamless physical-to-virtual movement mapping
- User stays centered on the treadmill during free navigation
- Foundation for future terrain simulation and gesture recognition