Many people don’t seem to know the Set Matte effect plug-in exists, maybe because it officially exists only to provide compatibility with earlier projects. That seems to have changed since Set Matte was upgraded to support 32-bpc color depth in After Effects CS6. Perhaps going directly to Trish and Chris Meyer’s explanation (free in 2019) is the best route, but here goes.
Set Matte was the go-to filter before Track Mattes appeared in the Time Layout window in After Effects 3.0, the first version shipped by Adobe. It’s a useful item if you want to use the same matte in multiple places, which is one of the main “use cases” that people mention when they get specific about nodal compositing needs (though not the likes of Maschwitz and Christiansen).
As noted in AE Help: The layer used as the matte with the Set Matte effect can be anywhere in the layer stacking order, unlike a track matte layer, which must be directly above the matted layer in the layer stacking order. Also, one layer can be used as the matte for multiple layers with the Set Matte effect.
An additional benefit of Set Matte over Track Matte (see Chris Meyer for background) is that you can add further effects to the main layer, an edge effect like Drop Shadow, or Remove Color Matting (for premultiplied alphas), Levels, etc.
A disadvantage is that Set Matte is a “compound effect” in the Rendering Pipeline (see videos by Angie Taylor and Chris Meyer), using the calculated result of a layer to modify another, so that masks, effects and geometric transforms must be applied to the matte layer first.
As usual, when in doubt, pre-compose.
Another shortcoming that may not be obvious, when reviewing a project, is which layer is the matte used by Set Matte, especially when scanning the Flowchart panel (though the effect name can be customized).
>>>> Perhaps the first stop in understanding should be the After Effects Classic Course: Set Matte versus Track Matte by Chris and Trish Meyer. Really, go read the text too!
By the way, Mikey Borup added a reminder, After Effects Quick Tip: Set Matte Effect,
Perhaps Adobe could add some extra juice to the Set Matte effect, like the extras seen in bs_SmartSetMatte, a Pixel Bender and now native plug-in by Bartek Skorupa. Alpha channels, track mattes, and channel effects like Set Matte, Calculations, and Shift Channels are explained in depth in the two main AE resources: Creating Motion Graphics for After Effects by Chris and Trish Meyer and the newly released Adobe After Effects CS6 Visual Effects and Compositing Studio Techniques by Mark Christiansen.
Chris & Trish Meyer’s series of tips, tricks, gotchas, factoids, and shortcuts continue with tips on Masking, Track Mattes, and Stencils in CMG Hidden Gems: Chapter 10 – All About Masking, CMG Hidden Gems: Chapter 11 – All About Track Mattes, and CMG Hidden Gems: Chapter 12 – Stencils and the “T” (the Transparency switch). For additional handy info and tutorials, see Track mattes and traveling mattes, Stencil modes, and Preserve underlying transparency in After Effects Help and The return of the Set Matte filter at AE Portal. If you don’t know already, check out the tip on hitting the Arrow keys while drawing Rounded Rect mask or Polygon/Star shapes. For some handy info, see the sections on creating and working with masks in After Effects Help, and posts tagged mask at AE Portal.
Both the Christiansen and Meyer’s books also detail wrinkles in the render pipeline, which necessitate a varied strategy for procedural matte creation. CMG reminds you throughout with render pipeline sidebars and flowcharts; important points can be found in the pretty new PDF excerpt from the Christiansen book, Chapter 4 Optimize Projects (under sample content).
Creating Motion Graphics is the best single resource for Set Matte information and includes tutorial and demo project files.
Set Matte does exist in Premiere Pro, but it’s for AE compatibility; the Track Matte Key effect does much the same thing and is hardware-accelerated (with CUDA at least). NLE Ninja has a tutorial using Set Matte in Premiere (at 26:26) and Eran Stern discusses how to overcome the limitations of the Track Matte effect using advanced tips and tricks in Compositing in Premiere Pro.
There’s been a few more tutorials since this was created, but they add little, though you may prefer the presentation. They may leave things out.
Jake in Motion, 2021, Set Matte | Effects of After Effects