Augmented reality (AR) is distinct from virtual reality (VR) in that it brings the real world into virtual gameplay. The most famous example of AR is Pokémon Go, which lets players find the pocket monsters throughout their own physical region. Minecraft is the best-selling video game of all time, but lacks any official AR gameplay. So Ryan Chan tackled the problem himself and built a system that translates real world movement into control of a player’s Minecraft avatar.

We’ll just assume that you know how Minecraft works, because you have probably played it yourself. Chan’s project works with the standard game and doesn’t require any special mods — Chan could even use this to play on others’ Minecraft servers if he chose. The system counts footsteps and converts them into forward movement in-game. It also detects real life rotational movement and replicates that movement in the game. But other actions, like attacking or swapping items, require conventional button presses.

The key components of this project are an Arduino MKR Zero board, a MPU-6050 IMU (inertial measurement unit), and two force sensitive resistors. The IMU detects rotational movement, while the force sensitive resistors detect footsteps when worn on the player’s shoes. Four mechanical key switches trigger the other actions. Chan configured the Arduino to appear as a standard USB HID keyboard and mouse when plugged into a computer, so Minecraft accepts the control commands without issue. To tidy everything up, Chan designed a custom PCB that hosts the aforementioned components.

Using this system for Minecraft gaming is tricky, as it requires plenty of real world open space to navigate the virtual world. But with access to a large park, it lets the player enjoy an AR Minecraft experience.