The reveal during the Mobile World Congress, in Barcelona, showcases the multi-year collaboration between Prophesee and Qualcomm resulting in next generation blur-free mobile photography, now ready for customer integration into Snapdragon 8 Gen 3 enabled devices. As ProVideo Coalition revealed last March, Prophesee had a goal: to optimize neuromorphic vision technologies for the next generation of smartphones, meaning no more blurred photos or videos.
The company developed what it calls a breakthrough Event-Based Vision approach to computer vision. This new vision category allows for significant reductions of power, latency and data processing requirements to reveal what was invisible to traditional frame-based sensors until now. Prophesee’s patented Metavision sensors and AI mimic how the human eye and brain work to dramatically improve efficiency in areas such as industrial automation, mobile, IoT, and AR/VR.
Now, a live demo during Mobile World Congress Barcelona is showcasing Prophesee’s native compatibility with premium Snapdragon mobile platforms, bringing the speed, efficiency, and quality of neuromorphic-enabled vision to cameras in mobile devices. And we can expect to see the technology appearing in – high-end – smartphones, including those from Xiaomi, as the company is one of the investors in Prophesee, a Paris based company that holds more than 50 international patents.
The new Prophesee’s event-based Metavision sensors and AI, optimized for use with Snapdragon platforms now brings motion blur cancellation and overall image quality to unprecedented levels, especially in the most challenging scenarios faced by conventional frame-based RGB sensors, fast-moving and low-light scenes.
“We have made significant progress since we announced this collaboration in February 2023, achieving the technical milestones that demonstrate the impressive impact on image quality our event-based technology has in mobile devices containing Snapdragon mobile platforms. As a result, our Metavision Deblur solution has now reached production readiness,” said Luca Verre, CEO and co-founder of Prophesee. “We look forward to unleashing the next generation of Smartphone’s photography and video with Prophesee’s Metavision.”
“Qualcomm Technologies is thrilled to continue our strong collaboration with Prophesee, joining efforts to efficiently optimize Prophesee’s event-based Metavision technology for use with our flagship Snapdragon 8 Gen 3 Mobile Platform. This will deliver significant enhancements to image quality and bring new features enabled by event cameras’ shutter-free capability to devices powered by Snapdragon mobile platforms,” said Judd Heape, VP of Product Management at Qualcomm Technologies, Inc.
Prophesee’s breakthrough sensors add a new sensing dimension to mobile photography. They change the paradigm in traditional image capture by focusing only on changes in a scene, pixel by pixel, continuously, at extreme speeds.
Each pixel in the Metavision sensor embeds a logic core, enabling it to act as a neuron.
They each activate themselves intelligently and asynchronously depending on the amount of photons they sense. A pixel activating itself is called an event. In essence, events are driven by the scene’s dynamics, not an arbitrary clock anymore, so the acquisition speed always matches the actual scene dynamics.
High-performance event-based deblurring is achieved by synchronizing a frame-based and Prophesee’s event-based sensor. The system then fills the gaps between and inside the frames with microsecond events to algorithmically extract pure motion information and repair motion blur.
Now, this innovative technology will mean something different for those who love to have blurry photos: they will have to deactivate the function for those artistic long-exposures…