My friend Gary Yost is a filmmaker, photographer, software developer, and lover of the natural world with a very unique blend of artistic and technical talents. His current passion project involves pulling footage from a network of remote cameras and turning it into something quite remarkable. But I’m going to let him explain it in his own words: everything that follows was written by Gary (with some light edits by me, so any typos or grammatical errors are my doing). -Mark
This article describes a technique for creating high-quality time-lapse videos by using a networked remote surveillance camera system. Built with tens of millions of dollars in infrastructure, this network was originally developed to assist in firespotting and firefighting efforts.
This artistic usage is not a purpose for which the network was intended, but anyone can do this with After Effects, Topaz Video Enhance AI, a couple of After Effects plugins and a fair amount of computer horsepower. It opens up a new world of nature/climate media content creation and you never have to leave your home or studio.
This image data is ephemeral: it’s not stored and if you miss it you’re SOL. So a big part of it is understanding the meteorology of the various regions of the West and this connects you more deeply with nature. There’s a feeling of exhilaration…of being “on the hunt” and when I see an interesting weather pattern developing, I’ll make a note of it and return to that location a few times over the next half hour or so. The optimal period to capture each shot is 1 hour, so if you wait more than 20-30 minutes you’ve lost the opportunity.
In some ways it’s like having a team of 700 videographers always ready to shoot in the most remote locations, 24 hours a day. In other ways it’s more intimate than that because you know that nobody’s there and you’re “lurking” in a place where no person actually is.
It’s an infinite source of time-lapse material, especially during the winter months when the weather is interesting. Without that, the landscapes are just a blank canvas (to me, anyway). And during fire season, the imagery can be truly terrifying.
It all started with a group of kids (self-named “Forest Guards”) tackling the problem of climate catastrophe back in 2008. Their idea was to create a network of crowd-sourced fire lookout towers throughout the west so everyone could become fire spotters.
What became the ALERTWildfire system began as a joint project between the Nevada Seismological Laboratory and the Forest Guard team, a group of young students from Meadow Vista in Placer County, California. The Forest Guards won the Innovate Award at the Children’s Climate Action in Copenhagen, Denmark in 2009. Their brilliant idea was to seed the forest with wirelessly connected cameras to enable early wildfire detection. Their most innovative contribution was the added ingredient of using social media and the web to engage the public to stand guard over the forest – to become Forest Guards. Their team leader Heidi Buck and 6 young adults joined together with the seismology lab and Sony Europe to deploy a prototype system in 2010.
By mid 2013, the Nevada Seismological Lab embraced a new generation of IP-capable, near-infrared HD cameras. These cameras were a game-changer, enabling a next-generation approach to early wildfire detection, confirmation and tracking. A mix of private and public funding provided the means to launch ALERT Tahoe and the BLM Wildland Fire Camera program. Recent funding through regional utilities, county-based organizations and CALFIRE enabled the vision set out by the Forest Guard young adults: nearly 1000 cameras in California alone, and eventually thousands in the Western US!
A decade later, the 5th generation of this system has been installed around the west, providing critical information on 1500+ fires in the past five years. The public now has access to this expanding system (www.alertwildfire.org), making their dream of a socially engaged public in the fight against wildfire come true. In Orange County alone, more than 400 citizens have volunteered to watch ALERTWildfire cameras during fire season, especially during red flag days and nights!
Here is a documentary made in 2009 by John Keitel & Stephen Kijak about the original Forest Guards.
And here are a few of the many articles about the history of the program:
The ALERTWildfire network has become a wide-ranging system with unique capabilities for visualizing some of the most remote areas of California (with a few additional stations in Nevada, Oregon, Washington, Idaho and Utah). Its primary purpose is to enable a new form of citizen science so we can all contribute to the safety of our forests and our atmosphere. But it can also be used for other purposes…and I’m using it to showcase California’s natural beauty.
My own background as a fire watcher goes back to the same year when the Forest Guards came up with their concept, in 2008. That was the year I became a fire lookout on Mt. Tamalpais, overlooking Marin County just north of the Golden Gate Bridge. My film “A Day in the Life of a Fire Lookout” became a Vimeo Staff pick and launched me as a filmmaker. After five years of active duty on Mt. Tam I became too busy to pull shifts but missed the solitude and the beautiful skies which I’d spent all day and night appreciating (and capturing with time-lapse).
In 2016 I discovered the ALERTWildfire network of cameras and began spending my mornings with a cup of tea browsing through images of beautiful locations in California, always keeping an eye out for the telltale sign of a smoke start. The ALERT network implemented a rudimentary timelapse feature that allowed for looking back over the previous 12 hours to get a sense of the smoke dynamics, start time, etc. Once the fire season would end in late autumn there wasn’t anything for me to do on the system but appreciate the beautiful winter skies.
There were uncountable times I wanted to share the beauty of what I was seeing with my friends but there’s no function to save the image stream. My friend Andy Murdock recommended I try using OBS to capture the screen output…perfect! My 2019 Mac Pro with 16 processors and dual Radeon cards is plenty powerful enough to run OBS, but the bigger challenge was that the Alert system’s sequential video images are not pushed out via a real-time codec…they’re literally individual low-res, highly-compressed jpegs that are sent down the wire from AWS. There are tons of duplicate frames in the stream with random frequency and duration, making it impossible to work with them further in post.
Here is the workflow I developed to capture, convert to a clean codec, deflicker, noise reduce, detail enhance, motion smooth, and edit these streams. Note that none of this would be necessary if we could actually pull real video off the system, but ALERTWildfire wasn’t built for that… it provides just enough time lapse imagery so a fire spotter can see how a smoke (or fire) develops over time. So it requires a bit of patience just like in the old days when we used darkrooms and many chemical steps to develop film…this is the modern-day equivalent.
This video provides an example of some of the most important steps. The first sequence is the tediously long, randomly stuttering footage that comes straight out of OBS (with the partner logo cropped out)… you’ll see how choppy it is. The second sequence is the duplicate frames pruned by Duplicate Frame Remover in After Effects; the third is after two passes in Topaz Video Enhance AI: Artemis and Chronos Fast; and fourth is with a LUT added for contrast/color along with final output sharpening.
Step 1: Capture
There are many ways to do screen capture but I use Open Broadcaster Software (OBS) to scrape the data off the ALERTWildfire network because it provides for discreet capture of one particular browser window as a background process instead of a standard screen capture tool, which would monopolize my system.
I run the sequence once through in the OBS browser before capturing to cache it as best as possible. Caching is important because you’ll notice that the first time you run it, the timeline sometimes jumps around and even goes backwards. After running it through once, I start playback ASAP after clicking on the Record Screen button in OBS to create a sequence with the shortest possible heads/tails. There will be duplicate frames and a few short pauses of 5-6 frames each but you’ll deal with these in Step 3.
Step 2: Trimming
Bring the captured files into your NLE (I’m using Final Cut Pro), crop them (I remove the sponsor logo but leave in the important status line at the bottom which shows the camera location, date and time). Trim the heads and tails to get rid of the OBS static frames, save as 30fps, 1920×1080 Prores Proxy or LT into a /4Pruning folder.
Left side as-captured from OBS – Right side cropped to remove sponsor logo, status line retained.
Step 3: Pruning – Removing Duplicate Frames
It’s essential to remove all duplicate frames so the next steps in Flicker Free and Topaz Video Enhance AI can function correctly. There are random numbers of duplicate frames so simply changing the frame rate in AE won’t work.
There are two ways to do this: manually in the NLE, or with a plugin for After Effects called Duplicate Frame Remover by Lloyd Alvarez, available on AEScripts. When DFR works it’s great, but you can’t count on it, especially for the more-subtle scenes (that also can be the most beautiful). DFR is simple to use; import all the /4Pruning files into Adobe After Effects, making individual comps for each of them using the select all/make comp function and then manually running DFR by selecting it from the Window menu. These need to be run manually due to not having batch capability for this plugin.
I always leave the DFR settings at default because I haven’t been able to get better results with either the Threshold or Area of Interest function.
As mentioned above, in a number of cases DFR doesn’t see the contrast changes from frame-to-frame correctly, especially if they’re subtle, and sometimes the ALERT system streams a very coarse set of frames; in these cases DFR will produce an undesirably short and choppy result. But I always run everything through DFR first because it’s impossible to know if it’ll work without actually trying it. If too many frames are removed I go back to the pruned file in the NLE, speed up the sequence 300-400% (which gets rid of the “on2s, on3s and if necessary the on4s”) and then I look for the small number of >4 duplicate frames that are left (identified by short pauses) and blade those out. This doesn’t take long and provides a decent result that you can further smooth out in Step 7.
Save all the clips to a /Pruned folder. Look shots that are “over-pruned” by DFR (easy to identify because of their small file size) and bring the /Pruned versions into the NLE for the manual process described above.
Step 4: Deflicker
A large percentage of shots will have noticeable flicker due to the constantly varying aperture of the surveillance cams. I use Digital Anarchy’s Flicker Free plugin inside of After Effects with the below settings and it does a great job of deflickering the output. Bring all of the /Pruned files into AE, add Flicker Free to them and render them out to an /FF folder.
Flicker-Free Default Timelapse settings with Time Radius set to 10 frames.
Step 5: Noise reduction / Detail enhancement
Next, I import all the /FF files into Topaz VEAI for noise reduction and detail enhancement. I usually use the Artemis Low/Blurry setting with no grain because it retains the most detail even though it does have a tendency to create some halo artifacting around high contrast edges (like mountain ranges against sky), but it does a great job with noise reduction. For shots in which detail isn’t that important you can use Artemis Strong Dehalo, which provides a much creamier result. I output to the same HD resolution because scaling up, even though it would generate a bit more detail, would be brutal at the next (motion interpolation) phase. Artemis uses both GPUs in my Mac Pro and this phase goes quite quickly. I Save these to an /Artemis folder.
Before and after Artemis noise reduction and detail enhancement in Topaz Video Enhance AI (click image for full size version)
Step 6: Motion interpolation/smoothing
I Import all of the /Artemis files into VEAI and use the Chronos/Fast function to smooth the cloud motion. I typically slow them down by expanding to 1.5x length and interpolate further by transcoding to 120fps. Leaving at 1.5x/30fps can work if you have a less powerful machine, 60fps would be better than 30. VEAI/Chronos provides a better result than either Premiere or Final Cut Pro can do using their optical flow algorithms. This will use all available processors and takes anywhere from 30-60min per sequence on my 3.2 GHz 16-Core Xeon Mac Pro. Save these to a /Chronos folder.
Step 7: Assembly
Bring the fully-processed files from the /Chronos folder into the NLE for assembly and in some cases minimal retiming for aesthetic issues regarding cutting to your music track. Given the original choppy source, even with all the previous post processing there may be motion discontinuities and I blade the clip at those points, pull in the frames around the blade and add a dissolve to eliminate the discontinuity.
If any additional retiming causes optical flow errors I’ll use a frame-blended sequence for those bits and transition in and out of that during the portions that exhibit the errors, thereby eliminating them.
A minimal color correction pass is required to deal with some of the surveillance camera’s funky white balance issues. Then I add a LUT to bump contrast/color slightly and with a bit of final post-sharpening, Bob’s your uncle.
Ironically, this workflow to capture beauty uses the same set of cameras that focus on tracking our most nightmarish disasters (and avoiding those). And in that I see it as the Yang to the ALERTWildfire project’s Yin. My hope in generating this imagery is to make it part of a larger project to help raise awareness of the system in advance of the 2022 fire season and enlist many thousands more fire spotters once they see these intimate views of all the beautiful spaces we have here in California to protect.
One of the things that makes this imagery unique is that there are a number of unusual elements that you wouldn’t see in a normal time lapse sequence, like:
- Towers and support structures
- Low dynamic range artifacts (blown highlights)
- Crazy white balance that can be psychedelic at times
- Privacy screens: To provide privacy near cameras located within the Wildland Urban Interface (WUI), buildings and other structures have been blocked out from field of view by either pixelation or small grey boxes.
- Birds or other animals near the camera and sometimes sitting on the camera and supporting strutures.
These unusual elements add context to the nature of these images and provide further evidence that they are a part of a utility network.
To view all of the tests I’ve been making with this workflow, please enjoy the work in this Vimeo Showcase, and for updates you can follow the project on Instagram @alertlapse.
We can all raise awareness of the ALERTWildfire network, and how it’s not just useful for spotting wildfires, but also as an immense source for appreciating the beauty of our natural world here in the West.
Gary Yost is an American filmmaker and software designer, best known for leading the team that created Autodesk 3ds Max, the world’s most popular 3D visual effects production system. He is an award-winning filmmaker with unique expertise in the combined fields of computer imaging, filmmaking and immersive storytelling. His 2021 Emmy-nominated VR film “Inside COVID19” was created by the nonprofit he founded in 2018, to create immersive experiences of shared wisdom. www.wisdomvr.org