Released in 2015, the Apple Pencil is a technology-packed stylus that allows users to write on iPad screens with variations in pressure and angle — all while communicating with very low latencies. Nekhil Ravi and Shebin Jose Jacob of Coders Café were inspired by this piece of handheld tech to come up with their own pencil concept, except this one wouldn’t need a screen in order to function.

The pair’s writing utensil relies on recognizing certain gestures as letters, and once one has been detected, outputs the result over USB or Bluetooth® to the host device. They started by first gathering many samples of different letters and how they correlate to the change in motion on the Arduino Nano 33 BLE Sense’s built-in accelerometer. From here, they designed an impulse in the Edge Impulse Studio to extract spectral features from the time series accelerometer data and pass it to a classification Keras neural network. The resulting model could accurately determine the correct letter from each gesture, making it suitable for deployment back to the Nano 33 BLE Sense.

Before testing their new inferencing code on the hardware, a simple 3D-printed case was designed to fit around the board to look like the real Apple Pencil. Additionally, the team made a simple website that could receive data from the board over BLE and display the corresponding letter within the browser window. To see more about this project, you can watch their video below!