As robotics advance, the future could certainly involve humans and automated elements working together as a team. The question then becomes, how do you design such an interaction? A team of researchers from Purdue University attempt to provide a solution with their GhostAR system.

The setup records human movements for playback later in augmented reality, while a robotic partner is programmed to work around a “ghost” avatar. This enables a user to plan out how to collaborate with the robot and work out kinks before actually performing a task.

GhostAR’s hardware includes an Oculus Rift headset and IR LED tracking, along with actual robots used in development. Simulation hardware consists of a six-axis Tinkerkit Braccio robot, as well as an Arduino-controlled omni-wheel base that can mount either a robot an arm or a camera as needed.

More information on the project can be found in the team’s research paper.

With GhostX, whatever plan a user makes with the ghost form of the robot while wearing an augmented reality head mount is communicated to the real robot through a cloud connection – allowing both the user and robot to know what the other is doing as they perform a task.
The system also allows the user plan a task directly in time and space and without any programming knowledge.

First, the user acts out the human part of the task to be completed with a robot. The system then captures the human’s behavior and displays it to the user as an avatar ghost, representing the user’s presence in time and space.

Using the human ghost as a time-space reference, the user programs the robot via its own ghost to match up with the human’s role. The user and robot then perform the task as their ghosts did.