So, these guys throw me a feature needing some digital janitorial work. I spend a few days in Motion and Shake, hand-tracking in fixes atop super-wide-angle, wildly distorting, handheld shots with exposure changes in 'em, and faking up some greenscreen comps for stuff that didn't get shot. Cool. But there are also stuck pixels throughout: the show was shot with a pair of HD camcorders: one had a bright white pixel in the upper right side of its image, the other in the upper left side (this, apparently, is what "fair and balanced" is all about). What to do? FXScript!
FXScript is a programming language built into FCP; it's used in most of the CPU-driven video filters, generators, and transitions (e.g., most anything that isn't a GPU-driven FxPlug effect). Here's the kicker: anyone can play with it. There's a simple FXScript development environment called FXBuilder built into FCP itself, letting you view existing filters, create new ones, and test them interactively.
Dude, that still sounds like work!
It's not as intimidating as it seems; you won't break the Mac or trash any files if you mess up your code. When you try to save or run your script, FCP will tell you if there are any syntax errors, undefined variables, or the like, and if you encounter runtime errors, like trying to divide something by zero, FXScript will simply display an appropriate error message instead of crashing.
Unfortunately, documentation for FXScript is a bit sketchy: Apple's own Using FXScript (PDF, 1.2 MBytes) hasn't been updated since FCP 4, and while it's pretty good at listing all the functions available, it's lacking in practical examples and in detailed description of variables and parameters. Joe Maller's FXScript Reference is a useful supplement with user-discovered details about some of the stuff Apple doesn't document. The best tutorials out there are on Joe Maller's original FXScript Reference pages, last updated in 2002 but still relevant today (ignore the big text box telling you to just go to his newer site; the new site has updates on some of the newer additions, but the old site has all the rich, meaty detail... and a big tip of the hat to reader Andy for reminding me about it!).
Really, the best way to learn FXScript is to open existing scripts and learn from them—with "Using FXScript" and Joe's two sites handy, of course.
Why should I bother?
But, of course, you're probably wondering why you should bother. After all, there are lots of effects plug-ins already available (look at the sidebar on Joe's site for a list of plug-in vendors, many using FXScript to create their plug-ins; Joe Maller himself sells the well-regarded Joe's Filters).
There are three main reasons to fiddle with FXScript:
- You need a filter to do something that no one else has thought of, or you need a variation of an existing filter with a greater range, a different UI, different parameters, or the like.
- You enjoy working with image processing and want to see what you can do.
- You're a cheap bastard like me.
Not convinced? Lemme show you my pixel-masker script, which took me a couple of hours to write one day, then another fifteen minutes the following week when I wanted to change the user interface and fix a bug.
Dead pixels, stuck pixels, hotspots... whatever you call 'em, they're those nasty, always-on white photosites that develop on solid-state camera sensors as they age. Most cameras have a process to mask them internally, typically by overlaying their signal with the signal from the leftmost adjacent photosite, but sometimes these processes aren't user-adjustable, or the pixel-masking memory fills up, or something else happens to make the hot pixels visible.
In our case, we had two cameras, each with a single, bright white spot in its image. As the bulk of the feature occurs at night, these hotspots stand out like sore thumbs in the largely low-key pictures (assuming that a sore thumb stands out like an annoyingly bright thing in the darkness that, once seen, can't be ignored).
Furthermore, these bright white photosites imposed their brightness into images undergoing substantial edge enhancement ("detail"), so each bright spot had a slight dark halo. The pix were recorded as XDCAM HD, so this contrasty constellation caused codec conniptions in the form of flickering mosquito noise around the hotspots.
Something had to be done.
I needed something that was easy to apply, let me control the position and size of the dead-pixel mask easily, and let me choose the most appropriate masking function for the scene. This is what I came up with:
The Pixel Mask controls after masking a dead pixel.
- Location - a standard "point" control; clicking the + button puts a crosshair on the image, letting you drag the mask over a dead pixel.
- Horizontal & Vertical Trim - Point controls are great for initial positioning, but (unlike point controls in Motion) they don't allow for easy incremental tweaks—so I added two sliders to allow independent trimming of both the X and Y locations of the mask. I use Location to get the rough location right, then I fine-tune things with the trims.
- Height & Width - While a single dead photosite may only be 1x1 pixels in size, camera processing and compression may spread or smear its effect. Height and Width default to 2 & 2, just to make the darned thing visible, but we found on this show that setting Height to 8 and Width to something in the range of 3 to 5 was needed to properly mask the dead pixels as well as the compression artifacts scattered around them. These slider controls run the range from 1-100, but they're nonlinear controls biased to the low end, so you get finer control over smaller numbers (the Height control above is set to 8; the left half traverses the range of 1-8 while the right half runs from 8-100).
- Mask Source - I provide four choices for how the pixel mask is to be generated:
- Left Edge - like the built-in masking on many cameras, this choice simply takes the pixel(s) bordering the left edge of the area to be masked and replicates it (or them) across the area.
- Left & Right Edges - takes the average of the left and right border pixels on each line. This is a good choice when the predominant scene content near the dead pixel consists of horizontal lines
- Top & Bottom Edges - take the average of the top and bottom border pixels for each column. Best when the local scene content is predominantly vertical.
- All Four Edges - For each pixel, averages the border pixels to the left, right, above, and below. This is often the best choice overall, so I've made it the default.
- Use Test Color - to aid in initial positioning of the mask, it defaults to a white square. Once you've positioned it, uncheck this checkbox to let the mask operate the way it's supposed to—at which point, if all goes according to plan, it becomes nearly impossible to see.
- Test Color - if you're trying to stomp a dead pixel on a bright background, you can dial in a different test color with this standard color control.
In practice, you simply drop the filter on a clip you want to fix. Use the Location control to position the white square atop the area you want to mask, then uncheck "Use Test Color". Fiddle with the trims, height, and width to properly mask the hotspot; you want the smallest mask possible that adequately covers the hot pixel and any of its side effects. Stepping through the clip for a few frames is useful to double-check things; toggling the mask on and off (and/or toggling the test color on and off) is useful to verify position and effect.
You can also try using different pixel mask sources, in case the default "all four edges" blend causes a visible bump or blemish in the image.
Once you're happy, you're done. If you have more clips from the same camera to mask, drag the filter to the Favorites bin in the Effects browser, and give it a descriptive name (like "pixel mask - upper left" and pixel mask - upper right" for the two instances we used for our two cameras).
If you have multiple dead pixels in a scene, just apply multiple pixel mask filters, one for each.
A last note: if you've added other filters to the clip that modify its size or position, like the "Earthquake" filter, make sure Pixel Mask is higher up in the filter stack, so it gets processed before its target pixels get shaken around.
Next: FXBuilder, and how to create an FXScript...