From smartphones to DSLRs, all modern equipments seems to offer video capture as an option. It's only natural that people capture videos of all the things around them, many times in what can be called a first-person perspective, as those videos are captured from helmet, chest mounted or hanheld cameras, that offer extreme portability but little or none form of stabilization able to suppress the shaky results usually seen in the final movie.
There are ways to reduce shakiness in videos, at the editing stage, but they usually mean cropping part of the image, what may not be the best solution, as it means part of the scene will be cropped out. Also, for some video capture, like extreme action, the tools available in the editor may not be enough to remove shakiness. Although there are professional and complex ways to extract important scenes from a video, they are not available to the casual user. A simple solution for some of those first-person videos is to increase the speed of the video, a time-lapse technique that seems to reduce shakiness, but still does not deliver a result as good as many would like. It is that same time-lapse technique that a Microsoft research team has used as the starting point for their development of a new technique.
Time-lapse versions of accelerated videos are normally called hyper-lapse, to emphasize that the camera is moving through space as well as accelerated through time. In the paper now presented the team, comprised of Johannes Kopf, Michael F. Cohen and Richard Szeliski, from Microsoft Research, shows how to create smooth hyper-lapse videos from casually captured first-person video.
The research team worked with cameras GoPro Hero2 and Hero3. All of the videos were taken in a single shot and captured at 29.97 frames per second with a resolution of 1280×960 pixels. The videos are between 3 and 13 minutes in length, and the output videos have a decimation (speed-up) factor of roughly 10, a value that provides significant speed-up while still providing enough context to follow the motion. As can be seen from the six videos, the technique does an excellent job of providing fluid camera motion while minimizing rendering artifacts. Although it is not perfect, it is a step in the right direction for better video from portable systems.
The trick to achieve this result is somehow easy to understand, although it's mathematical aspects are well beyond what most of us can follow. Knowing that “at high speed-up rates, simple frame sub-sampling coupled with existing video stabilization methods does not work, because the erratic camera shake present in first-person videos is amplified by the speed-up”, the team created “an algorithm that first reconstructs the 3D input camera path as well as dense, per-frame proxy geometries. We then optimize a novel camera path for the output video that passes near the input cameras while ensuring that the virtual camera looks in directions that can be rendered well from the input. Finally, we generate the novel smoothed, time-lapse video by rendering, stitching, and blending appropriately selected source frames for each output frame. We present a number of results for challenging videos that cannot be processed using traditional techniques”.