The general idea of this project was to combine a sub-GHz radio-connected MCU with a low-resolution thermographic camera (AMG8851) for distributed sensing and explore application scenarios.


At the core of the project is the AMG8851 thermophile sensor. It is an 8x8 pixel infrared array sensor with an I2C interface and a typical framerate of 10 fps. The sensor was combined with an Atmel328P microcontroller and a CC1101 transceiver to create a modular setup.


To test different application scenarios, we developed a prototype in Processing. It was used to read and visualize the sensor input and to forward the data via OSC to tools like the Gesture Recognition Toolkit (GRT).

data pipeline of the prototype

You can find a version of the application we used in the final installation on github.


Gesture tracking

We found several hand gestures that were consistently tracked- and classifiable with the GRT. The static display of one, two, or three fingers and an ok gesture were distinguishable enough in an indoor scenario.

thermal array overlay of a hand

thermal array overlay of a hand

Pose tracking

To understand the limits of the setup, we also tested different poses and distances in an outdoor scenario. The best results were captured within 1.5 meters of the sensor.

thermal array overlay of a person

thermal array overlay of a person

Environment tracking

We also recognized the potential of ambient heat emissions during the exploration, which opened up new ideas for interaction designs, e.g., positional tracking with the ceiling lights.

thermal array overlay of a door

thermal array overlay of a lamp

Public installation

Our final prototype resembles a sport-on-the-go installation to motivate people to engage in sportive activity during idle situations. The installation was tested at a bus stop, a plaza, a walkway, a corner scenario, and a traffic light. Interestingly people did engage with the exercise in public spaces, even though the norms of public spaces inhibit this behavior.

showcase of the final installation at a bus stop


  • Patrick Tobias Fischer (Supervisor)
  • Andreas Berst (Contributor: Hardware, Software, Documentation)
  • Kevin Schminnes (Contributor: Hardware, Software, Evaluation)