Multimodal control of a robot with a Microsoft Kinect camera



In this project an application was developed that could be used to control the industrial robot KUKA LBR iiwaa 7 R800 with gestures and voice. The sensors for detecting sound and images were provided by a customary Microsoft Kinect v2. The project evaluated what functionalities can be achieved with this technology.

Furthermore the calibration of camera and robot (hand-eye-calibration) needed to be as effortless as effortless as possible.


Space of time: 09/2015 – 02/2016

Coding Language: C# and Java

Team: alone

Background: Developed as a university project

Report (German): View Report

Bomb ‘n Tune gets honoured!


The game “Bomb ‘n tune”, that I developed together with my friend Lukas Fritsch got awarded with the “Gläserner Johahn”, an award for multimedia products given from the TU Ilmenau.

The price awards media project from students in three different categories (short movies, multimedia and media concept). We won the second price in the category multimedia.

Business Game Jam in Barcelona

Since we won the Indie Gameleon Game Jam 2015 in Hamburg, we got invited to the Jam Today Fair Business Jam in Barcelona.
Game Jammers from all over the world came together to brainstorm business and marketing plans for the gamejam games. In the end every team pitched their business plan in front of experienced people from the game-development-scene.
It was a great experience!

Bomb ‘n Tune


Second screen is conquering the video game scene. Big companies like Nintendo or Sony implemented such features in their latest consoles. Nintendo created the Wii U GamePad, which has an integrated display, while Sony uses the PS Vita to enable Remote Play.

The problem with both of the described systems is that you need special hardware to be able to use the second screen features. Nowadays nearly everybody possesses a smartphone, which has a multi core processor. So why don’t we use the hardware, that we are carrying around with us all the time, as a second screen device?

This project evaluates the use of a second screen game created with the HTML5 game engine Phaser.




In this game the smartphone acts as the controller (Player), while an other device with ideally a big screen acts as the spectator.
The game proceeds in two different phases.

Game phase:

The caroutput_Vt0xNR

Every player controls a car. Each car has the abilities to ignite a nitro and to fire a blast, that is blasting the other cars in the area away.

The ball output_Acm2e2

The ball as the key feature in the game needs to be hit by the cars. Each hit gives the player a point and a coin and increments the counter of the ball by one. The hundredth hit lets the ball explode and takes the cars nearby along. Exploded cars loose their points but keep their coins for the tuning phase.

Trapdoors output_DtcNKe

The level of chaos is increased by trapdoors, that are spread over the map. The trapdoor open in irregular intervals. Players, that are falling into open trapdoors, loose their points but keep their coins for the tuning phase.


The bumpers, that are located in the corners of the map, let the ball rebound and ensure, that the ball does not get stuck in the corners.

Tuning Phase:

After the ball is exploded or if any player but one is fallen into a trapdoor, the second game phase, the tuning phase begins. In the tuning phase each player can improve the following features of his car:

Speed: Tuning the speed  increases the top speed of the car.
Handling: Handling determines the turn speed and the acceleration of the car.
Armor: Increasing the armor value improves the behavior of a car, when it is hit by a blast.
Ammunition: Ammunition is the amount of blasts a player can shoot during one round.
Boost: The boost value is the amount of  nitro boost a player can use during one round.

After twenty seconds the next game phase starts automatically.


racer_diagramThe game was programmed using the JavaScript framework Meteor and the HTML5 game engine Phaser. On the server meteor enables the use of Node.js and has integrated support for WebSocket communication between the server and the clients.

With HTML5 you have the possibility to use the hardware sensors of devices like smartphones. It is possible to get information of, for example, the accelerometer. In this project the accelerometer was used to detect a shake of the device. Another HTML5 feature, that was used was the Vibration API to give the user feedback for specific events. Unfortunately HTML5 is not supported equally by all browsers. The Virbations API, for example, is not working on the Safari Browser for iOS devices. This leads to an inconsistent game behavior on different devices and can only be solved by finding exceptions for not supported features. But it can be assumed that the support for HTML5 features will improve with every upcoming browser update.

Phaser is an open source framework to create Canvas or WebGL powered HTML5 browser games. The focus on mobile browsers ensures that every available feature works good on a mobile device. Because it is open source and comes as one single JavaScript file it is easily expandable or can be easily integrated in other frameworks. The three build-in physics engines (Arcade, Ninja or P2.JS) give the developer the freedom to choose the best fitting physics behavior. In this project P2.JS was used since it is the most complex of the three engines and gives the most freedom for the development.


The game is presented in a cartoon style and uses an isometric projection as the perspective. The isometric projection enables a visual representation of three dimensions where technically only two dimensions are possible.

The car was created with the 3D graphics and animation software Blender. A turnaround was animated and exported as single frames. Those frame where combined to a sprite sheet with the tool Texturepacker.
Since the game is playable with up to 30 players it is necessary to have 30 different colored cars. The car was exported in grayscale and later on tinted dynamically in the program code.




Other game elements like the traps or bumpers where animated using Adobe Flash and also packed to a sprite sheet with Texturepacker. To complete the cartoon look a font was created and could be used as bitmap text in the game.



Nine Men’s Morris on a KUKA LBR IIWA 7 R800


Today’s important topic of research is robotics. Robots are used in many different industries. In the area of robotics one of the research areas is Human Robot Interaction (HRI). HRI is important for any robotics application where a human and a robot are both actuators. It is important that the HRI of a robot application is satisfying for the human so that no frustration arises while using the application.
A good way of developing or researching in the area of HRI is to develop games, since the interaction in games is one of the key factors. By developing an application where a human is playing with or against a robot, which is acting as a second player and tries to be as human-like as possible while playing, different approaches of HRI can be researched and evaluated.
In this project an application was developed that allowed a human player play the strategy board game “Nine Men’s Morris” against a robot. To enable the robot to sense its environment a camera was used to analyze the game area.

The robot

The used robot was the KUKA LBR iiwa 7 R800, which is very light weight (23.9 Kg) robot and can be used for intelligent industrial work. The seven axis of the robot allow a flexible movement in all directions. The robot has integrated sensors to simplify the implementation of force detection.

The robot can be programmed with the programming language Java and comes with a special IDE called KUKA Sunrise Workbench. Sunrise Workbench can also be used to load applications to the robot or to debug an application.

The camera

For visual analysis a Cognex In-Sight 7000 Integrated Vision System was used. The size of camera is 75mm x 55mm x 47mm and the connectors are Industrial M12 connectors. The robust and intelligent camera can analyze an image by its own. The communication to other devices can be implemented easily, because the camera can be for example set up as a Modbus/TCP server that can be reached by any device in the same network. To create analysis jobs the software “In-Sight Explorer” can be used. The software provides all features of the camera in a easy understandable user interface.



The robot is mounted on the table. It is connected via a LAN cable to a switch. The camera is attached to a beam at a fixed position and is also connected to the same switch. This way the robot and the camera will be in the same network and can communicate with each other.

 GitHub Repository      GitHub-Mark-120px-plus



Newsletter-Creator is a tool for users, who would like to create and manage newsletter campaigns. The tool allows users to manage lists of subscribers, create new newsletter campaigns from templates or with the drag and drop editor and schedule date and time to send the newsletter. Also same newsletter campaigns can be edited by several people at the same time.



Space of time: 09/2014 – 01/2015

Coding Language: Meteor (JavaScript, mongoDB), HTML, CSS (Bootstrap)

Team: 5 people

Background: Developed as a project during my master studies.

InnoGames Game Jam #7


The next InnoGames Game Jam in Hamburg was set on the last weekend.
The theme was a quote by Walt Disney “It’s kind of fun to do the impossible”.

Two astronauts are dreaming the dream of being the first person on the moon. Who will win? Who can start a rocket at first? Is it even possible to leave the planet? Or will your opponent destroy your rocket before it’s ready to launch? No one knows, but it’s still a lot of fun.

“To the Moon!” is a 2D multiplayer game for two astronauts fighting against each other. Trying to build their rockets to fly to the moon, they are stealing fuel and modules and shoot with their laser weapons.


Game title: “To The Moon!

Space of time: 14/11/2014 – 16/11/2014

Coding Language: Unity3D (C#)

Team: 7 people

created by: Control-25

Background: Developed at the seventh InnoGames Game Jam in 48 hours.

GitHub Repository      GitHub-Mark-120px-plus

You can play the game here