Site icon ProVideo Coalition

FXScript: FCP’s Most Under-appreciated Feature

FXScript: FCP's Most Under-appreciated Feature 8

FXScript: FCP's Most Under-appreciated Feature 1

So, these guys throw me a feature needing some digital janitorial work. I spend a few days in Motion and Shake, hand-tracking in fixes atop super-wide-angle, wildly distorting, handheld shots with exposure changes in ’em, and faking up some greenscreen comps for stuff that didn’t get shot. Cool. But there are also stuck pixels throughout: the show was shot with a pair of HD camcorders: one had a bright white pixel in the upper right side of its image, the other in the upper left side (this, apparently, is what “fair and balanced” is all about). What to do? FXScript!

FXScript?

FXScript is a programming language built into FCP; it’s used in most of the CPU-driven video filters, generators, and transitions (e.g., most anything that isn’t a GPU-driven FxPlug effect). Here’s the kicker: anyone can play with it. There’s a simple FXScript development environment called FXBuilder built into FCP itself, letting you view existing filters, create new ones, and test them interactively.

Working with FXScript is comparable to working with AppleScript, BASIC, JavaScript, Python, or the like: you can hunt-and-peck your way through it, though you’ll be better off if you have a general understanding of programming and image-processing concepts. There is a learning curve, but it’s easier to deal with than, say, Objective-C, the MVC paradigm, and the Cocoa frameworks.

Dude, that still sounds like work!

It’s not as intimidating as it seems; you won’t break the Mac or trash any files if you mess up your code. When you try to save or run your script, FCP will tell you if there are any syntax errors, undefined variables, or the like, and if you encounter runtime errors, like trying to divide something by zero, FXScript will simply display an appropriate error message instead of crashing.

Unfortunately, documentation for FXScript is a bit sketchy: Apple’s own Using FXScript (PDF, 1.2 MBytes) hasn’t been updated since FCP 4, and while it’s pretty good at listing all the functions available, it’s lacking in practical examples and in detailed description of variables and parameters. Joe Maller’s FXScript Reference is a useful supplement with user-discovered details about some of the stuff Apple doesn’t document. The best tutorials out there are on Joe Maller’s original FXScript Reference pages, last updated in 2002 but still relevant today (ignore the big text box telling you to just go to his newer site; the new site has updates on some of the newer additions, but the old site has all the rich, meaty detail… and a big tip of the hat to reader Andy for reminding me about it!).

Really, the best way to learn FXScript is to open existing scripts and learn from them—with “Using FXScript” and Joe’s two sites handy, of course.

Why should I bother?

But, of course, you’re probably wondering why you should bother. After all, there are lots of effects plug-ins already available (look at the sidebar on Joe’s site for a list of plug-in vendors, many using FXScript to create their plug-ins; Joe Maller himself sells the well-regarded Joe’s Filters).

There are three main reasons to fiddle with FXScript:

  1. You need a filter to do something that no one else has thought of, or you need a variation of an existing filter with a greater range, a different UI, different parameters, or the like.
  2. You enjoy working with image processing and want to see what you can do.
  3. You’re a cheap bastard like me.

Not convinced? Lemme show you my pixel-masker script, which took me a couple of hours to write one day, then another fifteen minutes the following week when I wanted to change the user interface and fix a bug.

The Problem

Dead pixels, stuck pixels, hotspots… whatever you call ’em, they’re those nasty, always-on white photosites that develop on solid-state camera sensors as they age. Most cameras have a process to mask them internally, typically by overlaying their signal with the signal from the leftmost adjacent photosite, but sometimes these processes aren’t user-adjustable, or the pixel-masking memory fills up, or something else happens to make the hot pixels visible.

In our case, we had two cameras, each with a single, bright white spot in its image. As the bulk of the feature occurs at night, these hotspots stand out like sore thumbs in the largely low-key pictures (assuming that a sore thumb stands out like an annoyingly bright thing in the darkness that, once seen, can’t be ignored).

Furthermore, these bright white photosites imposed their brightness into images undergoing substantial edge enhancement (“detail”), so each bright spot had a slight dark halo. The pix were recorded as XDCAM HD, so this contrasty constellation caused codec conniptions in the form of flickering mosquito noise around the hotspots.

Something had to be done.

The Solution

I needed something that was easy to apply, let me control the position and size of the dead-pixel mask easily, and let me choose the most appropriate masking function for the scene. This is what I came up with:

The Pixel Mask controls after masking a dead pixel.

In practice, you simply drop the filter on a clip you want to fix. Use the Location control to position the white square atop the area you want to mask, then uncheck “Use Test Color”. Fiddle with the trims, height, and width to properly mask the hotspot; you want the smallest mask possible that adequately covers the hot pixel and any of its side effects. Stepping through the clip for a few frames is useful to double-check things; toggling the mask on and off (and/or toggling the test color on and off) is useful to verify position and effect.

You can also try using different pixel mask sources, in case the default “all four edges” blend causes a visible bump or blemish in the image.

Once you’re happy, you’re done. If you have more clips from the same camera to mask, drag the filter to the Favorites bin in the Effects browser, and give it a descriptive name (like “pixel mask – upper left” and pixel mask – upper right” for the two instances we used for our two cameras).

If you have multiple dead pixels in a scene, just apply multiple pixel mask filters, one for each.

A last note: if you’ve added other filters to the clip that modify its size or position, like the “Earthquake” filter, make sure Pixel Mask is higher up in the filter stack, so it gets processed before its target pixels get shaken around.

Next: FXBuilder, and how to create an FXScript…


A Whirlwind Tour of FXBuilder

“OK”, you say, “I’m convinced. How do I make cool filters at home in my spare time, garnering the respect of my peers and the amorous attentions of my object of desire?”

Well, I can show you how to make cool filters at home in your spare time. The rest is up to you.

You can launch FXBuilder from FCP’s Tools menu, but that opens a blank editor window, as blank as your mind when you see the blank window blankly staring back at you. It’s easier and less intimidating to open an existing filter, transition, or generator in an editor; that way you have both a template to start building on, and an example of how it’s done. Indeed, one of the best ways to learn FXScript is to just open up any interesting-looking effects in editors and see how they’re written.

Select the Effects tab in the Browser and control-click on an effect for its context menu.

Select “Open in Editor”. Bam! Its secrets are laid open before your eyes!

(Mind you, this doesn’t work with every effect. Those with the Effect Class of “FxPlug” aren’t FXScripts and aren’t editable, and some 3rd-party FXScript effects are encoded to keep prying eyes out.)

When you have an editor open, a new FXBuilder menu appears in the menu bar:

The FXBuilder menu, with keying functions highlighted.

(If you don’t see the menu on your system, make sure the FXBuilder window is active.)

The menu lets you run the script in test mode, stop it, export the script as a text file, or save it as a plugin or an encoded (read-only) plugin. It also lists all the data types, functions, etc. that FXBuilder knows about: you can scroll through, for example, to Functions > Key > RGBColorKey, and if you click on it, FXBuilder inserts the following into your code:

RGBColorKey(srcImage, destImage, redTarget, redPass, greenTarget, greenPass, blueTarget, bluePass, softness, fillRGB)

If you look up RGBColorKey in “Using FXScript”, it says:

Fills either the alpha or RGB channels of the destination image buffer with a mask created by comparing the values of the pixels in the source image to the “pass” values and “target” numbers specified. “Softness” specifies the softness of the mask. “FillRGB” specifies whether the alpha or RGB channels are filled with the results.

Joe Maller’s FXScript Reference site doesn’t list any user comments for RGBColorKey; it only lists the parameters and says, “Related Links: BGDiff”.

That means that nobody’s seen the need to further explain the function or describe its parameters—which means it’s up to you to figure it out. You’ll observe that the parameter listing doesn’t say anything about what kind of parameters these are: Image buffers? Regions? Rectangles? Integers between 0-255? Floating-point numbers from 0.0-1.0? If you can’t figure it out from context, or by looking at other FXScripts, you just have to try something and see if it works.

Sometimes, folks have seen the need to add commentary. For the Point input control, “Using FXScript” simply says

input varName, “UIName”, Point, x, y
Creates a point entry control.

while the FXScript Reference says, at http://www.fxscriptreference.org/inputcontrols/point

Additional Notes
the point input control yields confusing values.

by stib on 13 Jun, 2004 08:36, Applies to: FCP 4.5 (HD) from: 203.164.168.184

don’t expect the point control to return pixel values, instead it returns the proportion of the frame from the centre. So a selecting point at the top right edge of frame yields {-0.5, 0.5}. Even though the user sees pixel values in the control box.

It’s quite clever really, but a bit confusing.

This is an especially useful clue since the numerical readouts in the Point input control (as used in the “Location” control of Pixel Mask) do show you the pixel values!

Really, that’s the hardest part about working with FXScript: the lack of info on the types and allowable ranges of all the various parameters. It’s a bit of a treasure hunt to find the information, but if you don’t mind the occasional treasure hunt and some trial-and-error experimentation, you can get a lot of cool things done relatively painlessly.

The FXBuilder window has two tabs: the FXBuilder Text Entry tab in which you edit your script, and the FXBuilder Input Controls. The Input Controls tab has two source wells into which you can drop one or two clips, generators, or stills to use as test sources. When you run your script, your scripts controls also appear in the tab, and a small, 320×240 test window appears, showing the first frame of your source(s) as processed by your script, and letting you play with the controls.

The Input Controls tab, and the test window.

For a lot of my Pixel Mask testing, where I wanted to see exactly what the edges of my mask were doing, I used the Two Color Ray (Video Generators > Render > Two Color Ray) as shown; other times I used a circular gradient, or bits of random clips (the actual clips I wanted to work with, when scaled down to 320×240, had their dead pixels shrunk down to invisibility, so they weren’t useful in test mode).

If you make a syntax error, FXBuilder will tell you when you try to run the script by highlighting the error in the Text Entry tab and giving you an appropriate error message. For example, if I have the code

point srcPoly[4], destPoly[4]

boundsOf(Src1, srcPoly)
boundsOf(Dest, point) // mistake: “point” should be “destPoly”

and I try to run the script it’s in, FXBuilder highlights the word “point” in the last line, and gives me a popup saying, “Error in FXBuilder script: FXScript Error: Type mismatch”. It’s then up to me to figure out what my “type mismatch” is, and fix it.

Once you’ve made your script do what you want, you have three choices:

  1. Create Plugin: make an FCP plugin that anyone can open, read, and edit.
  2. Create Encoded Plugin: make an FCP plugin that can’t be opened in FXBuilder.
  3. Export Text: make a plain-text file that anyone can open, read, and edit, and it still works as a plugin.

I tend to save things as text, so I can edit them in Text Edit or XCode as well as in FCP. if you’re selling your plugin, you may want to save it as an encoded plugin, but be sure to save an un-encoded version or a plain-text version as well, otherwise you won’t be able to edit it again! (Note: the encoding isn’t a perfect copy-protection scheme, either, so don’t depend on it to keep your secrets safe from determined safe-crackers.)

To use your plugin for real work, quit FCP, and drop the plugin in /Library/Application Support/Final Cut Pro System Support/Plugins (it’s common practice to put your plugins in their own folder inside the Plugins folder). Restart FCP, and your plugin should appear in Effects. Alas, you can’t just save the file into the Plugins folder while FCP is running and keep going; FCP won’t pick up the changes to the plugins until you restart it.

Next: Pixel Mask unmasked line-by-line, and where to get it…


If you’re so smart, geek-boy, show us your code!

OK, I will. Let’s have a walk-through, so those new to FXScript can see what’s involved, and you other geek-boys can leave snide comments about how inefficient my solution to the problem is—or correct me gently where my assumptions and interpretations of the scanty FXScript docs has led me astray.

scriptid “Pixel Mask AJW” //DO NOT LOCALIZE
filter “Pixel Mask [ajw]”;
group “AJW’s Filters”;

The scriptid is a string that uniquely identified this script from all others. The “do not localize” comment, copied verbatim from every other FXScript I’ve seen, is a reminder not to translate this string into other languages (yes, I could have a version of this script with a French UI, or a Kurdish one, or Elvish or Klingon [note to self: make sure that the ProApps framework supports Klingonese characters before trying this], though the Plugins folder doesn’t have separate folder for different language versions).

The “filter” line tells FCP that (a) it’s a filter, not a generator or transition, and (b) what the displayed name of the filter is. It’s common practice, though not universal, to put some sort of identifying tag in the name (like the “[ajw]” suffix) to distinguish your filters from Apple’s own filters and other third-party filters: Joe Maller calls all his filters “Joe’s” whateveritis, Digital Heaven prefixes all their names with “DH_”, and so on.

“Group” tells FCP where to display the filter; in the user interface it appears as the submenu (in the Effects menu) or the bin (in the Browser) where the filter appears. Again, it’s common (and possibly universal) practice to put all your filters in your own group, so they don’t get mixed up with other third-party filters in FCP’s UI.

// hacked together 8 Feb 2009 by Adam J. Wilt, http://www.adamwilt.com/
// 15 Feb 2009 – fixed blit src bugs, opacity bugs; improved UI
// 16 March 2009 – changed default mask src to all 4 edges

// Use to hide stuck pixels

Comments are useful to track version changes and to explain what’s going on, like my detailed and extremely insightful comment, “Use to hide stuck pixels”. Well, d’oh!

input pixelLoc, “Location”, Point, 0, 0

input trimX, “Horizontal Trim”, slider, 0, -5, 5 snap -5, -4, -3, -2, -1, 0, 1, 2, 3, 4, 5
input trimY, “Vertical Trim”, slider, 0, -5, 5 snap -5, -4, -3, -2, -1, 0, 1, 2, 3, 4, 5

input pixelHeight, “Height”, slider, 2, 1, 100 ramp 95
input pixelWidth, “Width”, slider, 2, 1, 100 ramp 95

input blurType, “Mask Source”, Popup, 4, “Left Edge”, “Left & Right Edges”, “Top & Bottom Edges”, “All Four Edges”

input useCol, “Use Test Color”, checkbox, 1
input col, “Test Color”, Color, 0, 255, 255, 255
input unused, “¬© 2009 Adam J. Wilt”, label, “”

“Input” statements define the user interface that appears in the Filters tab of FCP’s Viewer:

Compare the controls shown here with the code that generated them.

Some notes:
– The “snap” parameter is supposed to keep sliders from taking on non-integer values, but I’m not sure if it’s actually useful. You can still type fractional values in the text entry field.
– “Ramp 95” is what gives the Height and Width sliders their nonlinear scales. Oddly, the tick marks on the displayed sliders compress in the wrong direction; they should bunch up at the high end where the numbers get closer together, instead of at the low end where numbers are farther apart! That’s just the way it is.
The blurType popup has a default value of 4, which causes the “All Four Edges” value to be displayed. I made that change today before uploading the filter to my website, because that’s the value I’ve been using the most. If you want a different default, you can edit your own copy to pick a different value.

InformationFlag(“YUVaware”)

InformationFlag is a hint to the system. “YUVaware” tells FCP that this filter can work without converting the frame buffer formats to RGB, so super-whites survive this filter and unneeded color-space conversions are avoided.

There are other InformationFlag values, mostly undocumented by Apple, which various people have uncovered; the best listing I’ve found is at http://www.fxscriptreference.org/search.php?search=InformationFlag

code

point srcpoly[4], destpoly[4]
float height, width
boundsOf(Src1, srcpoly);
boundsOf(Dest, destpoly);

The keyword “code” separates all the declarations and definitions above from the the stuff that actually does the work.

I define two point arrays to hold the corner points of source and destination rectangular regions (simple polygons), and fill them with the bounds of the system-defined Src1 (the source input frame, e.g. the frames of the clip this filter is applied to) and Dest (the frame that results from the action of the filter).

Many FXScripts start off this way: learning how big the frames are that they’re working on, and setting variables to match. That’s how the same filter can work on a 720×480 DV-format frame and a 1920×1080 HD frame with no size-specific user input needed.

// scale pixelLoc from the range 0-1 to 0-screensize
DimensionsOf(Src1, width, height)
pixelLoc.x *= width
pixelLoc.y *= height

// offset by half the size of affected area
pixelLoc.x -= (pixelWidth / 2) – 0.5 – trimX
pixelLoc.y -= (pixelHeight / 2) – 0.5 – trimY

pixelLoc.x = integer(pixelLoc.x)
pixelLoc.y = integer(pixelLoc.y)

Remember that comment about how a point variable is defined? We need to convert 0.0-1.0 numbers into an actual pixel location, so we scale ’em, offset ’em by half our mask size (so the mask is centered on our target point) and by our trim values, and then make ’em into integer values so we position our mask on whole pixels.

// copy src to dest
blitrect(src1, srcpoly, Dest, destpoly)

// define dest region for blit or fill
makerect(destpoly, pixelLoc.x, pixelLoc.y, pixelWidth, pixelHeight)

We blit (bit-block transfer, e.g. copy a rectangular block of bits or pixels) the source image into the destination buffer, since we want to have the source image output with only a small change around the dead pixel.

Then we compute the destination rectangle to be filled by our pixel mask based on the previously-computed corner location, and the height and width of the mask.

if useCol
    region reg
    MakeRegion(destpoly, reg)
    fillregion(reg, dest, col)

If we’re using the test color, simply fill in that destination rectangle with the test color, and we’re done.

If we’re using the real mask, things are a bit more complex… grab yourself a sandwich and a cup of tea, and dive in…

else
    // each blit layer should contribute opacity of (1 / layer) to the final composite
    float layer
    layer = 1

    // blurtypes:
    // 1 = left edge
    // 2 = left & right
    // 3 = top & bottom
    // 4 = left, right, top, bottom
    
    if blurType != 3    // not top & bottom only
        makerect(srcpoly, pixelLoc.x-1, pixelLoc.y, 0, pixelHeight)
        blitrect(src1, srcpoly, Dest, destpoly) // blitrect = blit with opacity of 1
        layer++
        if blurType == 2 or blurType == 4 // left & right, or all 4 edges
            makerect(srcpoly, pixelLoc.x+pixelWidth, pixelLoc.y, 0, pixelHeight)
            blit(src1, srcpoly, Dest, destpoly, 1 / layer)
            layer++
        end if
    end if
    if blurType >= 3 // includes top & bottom
        makerect(srcpoly, pixelLoc.x, pixelLoc.y-1, pixelWidth, 0)
        blit(src1, srcpoly, Dest, destpoly, 1 / layer)
        layer++

        makerect(srcpoly, pixelLoc.x, pixelLoc.y+pixelHeight, pixelWidth, 0)
        blit(src1, srcpoly, Dest, destpoly, 1 / layer)
    end if
end if

There, got that?

This block of code works by blitting the selected border edge pixels over the mask area. The first blit is done with an opacity of 1.0. The second blit (if performed) uses an opacity of 0.5: if we want to average the top and bottom border pixels, each should contribute half, so using a half-opacity second blit lets half the first blit combine with the second to average ’em out. If we do a third layer, its opacity wants to to be 1/3rd (letting 2/3 of the previous blits through, each of which is itself 1/2 the original value, thus all three layers contribute 1/3 the value of the final pixel), and so on: each new layer wants an opacity of 1 / (layer number).

Once you get that aspect, the rest of the block is simply parsing the blurType parameter to determine which direction(s) to blit new pixels from, what opacity to apply to them, and then performing the blit(s).

It’s worth noting that the source blocks for the blits have a width or height values of 0, not 1, for the narrow dimension: apparently the blit functions march across their regions using an algorithm like:

stepSize = width / numberOfSteps
for (x = startPixel, step = 0; step < numberOfSteps; step++, x += stepSize)...

so using a width value (or height value, where appropriate) of zero causes the source rectangle to be one pixel wide or high, not zero. This was the bug I fixed a week after writing the code; I had ones in the code, not zeroes, and I noticed that when testing along the diagonal edge of a Two Color Ray pattern that I was seeing a fade in pixel values across the mask, not consistent values: my source rectangles were two pixels wide (or high). The blit functions do an excellent job of interpolating pixels when stretching or shrinking a source rectangle to fill a destination rectangle!

Some side notes:

FXScript doesn’t seem to be case-sensitive.

Semicolons are only necessary as statement separators if you have multiple statements on the same line. As a C/C++ programmer for way too many years, though, they just pop out in my code whether I want them there or not.

New versions of FCP sometimes add new functions or InformationFlag parameters, and sometimes change the way existing functions work (or so folks like Joe Maller and Graeme Nattress tell me, though I haven’t had any of my filters break across versions). Apple doesn’t do a very good job at keeping people updated about the changes, either: the docs date from FCP 4 in 2003, fercryinoutloud. Hey, it could be worse; it could be coding for the web! FXScript doesn’t change that fast or that drastically, fortunately.

Dude, my brain like totally hurts now. Thanks for nothing, geek-boy!

Yeah, OK, maybe roll-your-own FXScript filters aren’t for everyone. At least you’ve gotten a glimpse into what coding is like, and consider that the long-suffering programmers on the Apple ProApps teams have to deal with this sort of thing—and worse—every day. Think of that the next time you’re dealing with some whiney client who can’t seem to decide what shade of fuschia his logo should be rendered in, and just give thanks that you aren’t having to grind code for a living!

If, however, you’re of the mindset that you want to fiddle with images in a programmatic manner, there’s probably no easier way to do so than through FXScript. It’s free with every copy of FCP. The official documentation (such as it is) is on Apple’s website, Joe Maller’s original FXScript Reference pages have many of the details Apple’s docs are missing and narratives of how he built his filters, while Joe Maller’s newer FXScript Reference fills in some of the gaps that have developed since 2002.

Have fun.

I came all this way, and I don’t even get a frackin’ T-shirt?

What do I look like, CafePress or something? Sorry, Binky, no T-shirts ’round here!

What I do have for you is a bundle of FXScript filters. Quit your griping, they’re free, aren’t they?

http://www.adamwilt.com/downloads/AJW’s Filters.zip.

These include :

Download them and unzip them; you’ll get a folder called “AJW’s Filters”. Drop it into /Library/Application Support/Final Cut Pro System Support/Plugins. Start (or restart) FCP. Hey presto: new filters! Use them, take them apart, rebuild them, modify them for your own nefarious needs based on your own brilliant brainstorms. Throw some or all of them away if they bore you. That’s freedom, isn’t it?

Uh, what were those references again?

Apple’s official document: Using FXScript, a 1.2 Mbyte PDF.

Joe Maller’s tutorials and notes: FXScript Reference for Final Cut Pro.

Joe Maller’s Web 2.0-esque FXScript Reference for post-2002 updates. (Look at the sidebar on the right side of the homepage for various vendors of FXScript filters.)

Exit mobile version