Human Motion Capture and Recognition

Model-based markerless upper body tracking.
Marker-based human motion capture with Vicon.

Markerless Human Motion Capture

Markerless human motion capture is a prerequisite for imitation learning as well as for human robot interaction. Our research focusses on real-time stereo-based methods that utilize the stereo camera system of typical humanoid robot heads, having a baseline comparable to human eye distance. As statistical framework a particle filter is used.
The listed publications illustrate the development and the advances of our approach, starting with a monocular approach, then introducing 3D hand/head tracking as a separate cue, and in our most recent work incorporating inverse kinematics, adaptive shoulder positions to allow for more flexibility of the model, and a prioritization scheme for cooperative cue fusion. Currently, we can capture 3D human motion with the head of the humanoid robot ARMAR-III in real-time with a processing rate of 20 Hz using conventional hardware. The captured trajectories are used for online reproduction.

Marker-Based Human Motion Capture (Vicon)

Marker-based human motion capture frameworks are widespread systems in the robotics research community as well as in the industry. One of the most popular commercially available systems is provided by Vicon. The technique, which is used here, relies on infrared cameras and artificial reflective markers. The markers are placed on predefined body parts of a human subject. In a defined workspace, the subject is surrounded by a set of infrared cameras. Each camera is equipped with a infrared strobe, emitting a light signal, which is reflected by the markers. The reflected light,
which distinguishes itself from the background, is registered by the cameras. The data from each camera consisting of 2D coordinates of each recognized marker position, is merged in a data station, which computes the 3D position by triangulation and the label of each visible marker. Due to the high-speed and high-resolution properties of the cameras, the Vicon system provides an accurate method for capturing human motion at high frame rates.