Blog Home

Control with your smart devices by staring and gesturing

Arduino TeamJuly 26th, 2016

As part of a recent Microsoft HoloLens hackathon in San Francisco, Maker Ian Sterling developed a new app that interacts with you smart home via augmented reality. The proof of concept, dubbed “IoTxMR,” allows a user to simply glance at a gadget and control it through gestures.

As you can see in the video below, IoTxMR enables Sterling to connect various Android and Arduino-based devices with the HoloLens to create a customized interdependent network. It also features a mixed reality experience called “virtual zen mode,” complete with calming sounds and light orbs in his surrounding environment.

During a recent interview with Digital Trends, Sterling revealed:

The primary goal of the app is to provide a 3D spatial UI for cross-platform devices — Android Music Player app and Arduino-controlled fan and light — and to interact with them using gaze and gesture control.

The connectivity between Arduino and a mixed reality device is something which holds a huge amount of creative opportunity for developers to create some very exciting applications — be it [Internet of Things], robotics, or other sensor data visualization. Besides this, our app features some fun ways to connect devices. Our demo featured a connection between a music player and a light in order to set a certain mood in your home.

Although just a demo, IoTxMR does highlight the endless possibilities that AR platforms like HoloLens offer in the not-too-distant future.

Categories:ArduinoFeatured