Multimodal control of a robot with a Microsoft Kinect camera

3d

Description

In this project an application was developed that could be used to control the industrial robot KUKA LBR iiwaa 7 R800 with gestures and voice. The sensors for detecting sound and images were provided by a customary Microsoft Kinect v2. The project evaluated what functionalities can be achieved with this technology.

Furthermore the calibration of camera and robot (hand-eye-calibration) needed to be as effortless as effortless as possible.

Information

Space of time: 09/2015 – 02/2016

Coding Language: C# and Java

Team: alone

Background: Developed as a university project

Report (German): View Report