Axis Communications: making music with surveillance cameras

Axis is the company behind the world’s first musical performance played entirely on AI-powered video surveillance cameras and speakers pushing the limits of what surveillance technology can be used for.

Back in January 2024 ProVideo Coalition revealed how a network camera from Axis was frozen below an ice hockey rink to capture an angle of the sport never seen before. Now it’s time to follow the company into another adventure, as Axis Communications pairs AI-based analytics and video surveillance cameras to create a fully playable musical instrument.

Pairing high-performance video surveillance cameras with AI-based analytics, Axis Communications created a new kind of instrument that reads everyday objects like tennis balls, coffee cups and fire extinguishers as musical cues. As a predetermined item moves across the camera’s field of view, different zones trigger different notes, turning a video surveillance camera into a fully functional musical instrument.

“With this experiment, we wanted to challenge what is possible with our video surveillance cameras and AI-based analytics when it comes to detecting custom objects and triggering actions in real time”, says Ghaith Sankari, Experienced Software Engineer at Axis Communications.

The piece performed is a reimagining of Richard Strauss’s Also Sprach Zarathustra, made iconic by its use in 2001: A Space Odyssey. But instead of strings and horns, this version uses tennis balls, a coffee cup, and a fire extinguisher.

“We could have used any objects really, even sounds can trigger actions. What you train the cameras and analytics to detect and what you make them do is totally up to you”, says Sankari.

The musical arrangement is created by Swedish music producer Jonas Quant, known for his work with Kylie Minogue, No Doubt, and Hurts.

“What fascinated me”, says Quant, “was treating object detection itself as a new kind of instrument. Since we’re not using a keyboard this experiment posed some interesting creative challenges. Like teaching the musicians how to play an invisible instrument in thin air. “

The instrument works by detecting specific objects in zones assigned with a note. As the camera detects the right object it sends a MQTT signal to a MIDI that plays a sound. Each note was represented as a square within the camera frame. To perform the reimagined Also Sprach Zarathustra we needed four musicians playing four cameras connected to a computer and a live mixer board.  ~

“By combining AI-based object detection, real-time MQTT signaling and integration with external systems, in this case MIDI and synthesizers, we have transformed surveillance hardware into a responsive, programmable system. It’s a compelling proof-of-concept for how intelligent network technology can unlock new business applications far beyond traditional security”, says Ghaith Sankari at Axis Communications.

While unexpected and playful, the object detection orchestra is more than a novelty. The same technology that makes this performance possible is also powering automation, enabling smart cities, and shaping the future of machine interaction, taking video surveillance far beyond security.