In parts 1-3 of the PsF’s missing workflow series, we introduced the terms benign PsF & malignant PsF, and revealed the PsF status of several professional AVCHD cameras from 3 manufacturers (Canon, Panasonic, and Sony). In #4, we did the same with several HD video recorders from 6 different manufacturers. In #5, we revealed how one recorder manufacturer is offering its own “Band-Aid” software to counteract the inappropriate signals offered over HDMI by many camera manufacturers. In #6, I published an open letter to all pro AVCHD manufacturers. In #7, I covered how to deal with PsF on a progressive sequence in Premiere Pro CS5.5. Now in #8, I’ll reveal how the US$49.99 middleware known as ClipWrap will be the cure-all for all of AVCHD’s multiple weaknesses, including both types of PsF, at least in the short term.
The terms Benign PsF and Malignant PsF
The terms benign PsF and malignant PsF were introduced in Part 1 of this series called PsF’s missing workflow.
A summary of AVCHD’s multiple weaknesses for anyone who plans to edit the footage
For all of the details, please read (or review) parts 1-7 of this PsF’s missing workflow. In the meantime, here is a short review of those AVCHD weaknesses:
- Current AVCHD cameras (even professional ones) have chosen to record using lame FAT32 formatting, which unnecessarily forces the creation of multiple sequential files for long clips, which is a mess to view and manage at a system (“Finder”) level, and (without special workarounds in the software editor) requires manual rejoining of said sequential clips.
- Due to its heritage in the Blu-ray format, instead of simply recording video files, all current AVCHD cameras (even professional ones) record video files in a strange hierarchy of folders and subfolders, which are also a mess to view and manage at a system (“Finder”) level.
- Also due to its heritage in the Blu-ray format, 1080/25p and 1080/29.97p are not specifically valid, so all current AVCHD cameras (even professional ones) are unfortunately forced to record 1080/25p as 1080/25PsF, and 1080/29.97p as 1080/29.97PsF. There is not consistency in the way the PsF files are flagged (identified) among AVCHD manufacturers, so even though some PsF files (benign PSF) are properly recognized as progressive by some very recent software editing programs (i.e. Adobe Premiere Pro CS5.5.x and Apple Final Cut Pro X), other PsF files (malignant PsF) are misinterpreted as interlaced (with all of the pitfalls and negative side effects mentioned in part 1 of this series), and all AVCHD PsF files (even benign PsF files) are subject to pitfalls and negative side effects with older software editing programs.
In part 6 of the PsF’s missing workflow, I requested all professional AVCHD camera manufacturers (Canon, Panasonic, and Sony) to offer a professional UDF option in their camera menus, where they would format their flash media as UDF and record standalone long-GOP H.264 files as a single file per clip (not sequential files per long clip), and without unnecessary folders and subfolders. (There were several other requests in that letter.) While I hope they add these improvements to their current or upcoming long-GOP H.264 camera models, ahead you’ll see a cure for all of the AVCHD weaknesses mentioned above, plus some more features too.
Like a bridge over troubled waters, ClipWrap will now be the cure-all for AVCHD’s multiple weaknesses for many Mac video editors, at least in the short term.
ClipWrap is what I call middleware, because it acts as a bridge between the mess of sequential files, subfolders and potentially misinterpreted PsF (25PsF and 29.97PsF) into straightforward files that are recognizable both by the latest video editing apps, as well as the older ones with no problem. As the name indicates, in its default mode, the ClipWrap program re-wraps the original entangled H.264 files into straightforward .mov files. Shortly after releasing the very first articles in the PsF’s missing workflow series, I contacted Colin McFadden at Divergent Media to see whether he could add support for malignant PsF files to their US$49.99 ClipWrap product (which already handled benign PsF files properly), since I knew that if he did, ClipWrap would go from being a very good product to a “grand slam”. I had previously read that Divergent Media was vehemently opposed to adding a de-interlacing feature to their ClipWrap product (which I applaud) so I was careful to explain in my request/suggestion that I was not requesting de-interlacing, but just proper identification of all types of PsF (benign and malignant) in the resulting file as progressive, while continuing to identify the truly interlaced files as interlaced. Colin requested an example file of malignant 29.97PsF which I immediately supplied (thanks to Jorge Koechlin and Rafael Alarcón for that footage). I was surprised how quickly Colin sent me a ßeta version of ClipWrap to try it. I determined that ClipWrap was now capable of properly identifying the following types of AVCHD files:
- Benign 25PsF clips from the Panasonic AG-AF100 (thanks to Rub©n Abruña, from Miami, Florida, USA)
- Benign 29.97PsF clips from the Panasonic AG-HMC40 (thanks to Carlos Matamoros, from Miami, Florida, USA)
- Benign 29.97PsF from the Sony HXR-NX5 (thanks to Sony)
- Malignant 29.97PsF clips from the Canon XA10 (thanks to Jorge Koechlin and Rafael Alarcón, from Miami, Florida, USA)
- Native 23.976p clips from the Sony HXR-NX5 (Thanks to Sony)
- Truly interlaced AVCHD files from various sources
I determined that the only type that ßeta version ClipWrap couldn’t yet recognize as progressive were the malignant 25PsF clips from Canon (PF25), so Colin asked me to send him a sample 25PsF file, which I did (thanks to Jorge Mezéi, from Buenos Aires, Argentina). After receiving it, Colin updated the ßeta version, and I subsequently verified that the updated version now recognized all types of AVCHD PsF files properly, and the resulting .mov files were recognized as progressive in all of the editing programs with which I tested it, including Adobe Premiere (even older versions) and FCP X. Although I don’t have Avid Media Composer, Colin does, so I trust that it appears properly there too. The only place where the re-wrapped H.264 files don’t appear as progressive was in VideoSpec (since VideoSpec looks at another part of the metadata), but the most important thing is that the progressive files be recognized and treated as progressive in the editing programs, as they now are.
But beyond the good news about the fact that as of public version 2.4.5 (and later) ClipWrap now properly handles all types of AVCHD with all types of PsF, let’s talk about ClipWrap’s other wonderful functions and benefits. If you are a Mac video editor who receives AVCHD footage with malignant PsF exclusively (or malignant PsF together with any other type of AVCHD footage), ClipWrap is the tool you should be using to transfer footage from the camera or flash media card to your disk array, for several reasons. Here are the nice things that ClipWrap will do (or can do, optionally) in the process of transferring your footage (in its default, rewrap-only mode):
- While transferring your footage to your disk array (or media drive), ClipWrap will create .mov files, without transcoding, without losing a generation, at the approximate speed of a file copy. In the process, ClipWrap fortunately eliminates all of AVCHD’s unnecessary folders and subfolders.
- If you have long clips which the AVCHD/FAT32 monster has forced into multiple sequential files, ClipWrap will automatically rejoin the spanned files from long clips into a single .mov file, as indicated above: without transcoding, without losing a generation, at the approximate speed of a file copy. And as stated above, in the process, ClipWrap eliminates all of the unnecessary folders and subfolders.
- If the raw AVCHD footage is native 23.976p, ClipWrap will leave it as such, creating a native H.264 file with a .mov extension, which will be properly recognized as progressive by your editing program.
- If the raw AVCHD footage is 25PsF or 29.97PsF (benign or malignant), ClipWrap will create a true 25p or 29.97p H.264 file with a .mov extension, which will be properly recognized as progressive by your editing program.
- If the raw AVCHD file is 50i or 59.94i, ClipWrap will create a corresponding interlaced H.264 file with a .mov extension, which will be properly recognized as interlaced by your editing program. If you later import them to a progressive sequence, your editing program will deinterlace it (as it should in that case). If you later import them to an interlaced sequence, your editing program will leave it alone (as it should in that case).
- If your raw AVCHD footage has AC-3 audio, ClipWrap can optionally convert the audio to LPCM, while leaving the video alone. This is an option in ClipWrap’s preferences, and can be helpful in two cases: 1) If your editing program doesn’t support AC-3 audio well (or at all). 2) If the original AC-3 recording is multichannel (surround sound) but your editor (i.e. FCP X) automatically mixes them down to stereo and you really wanted discrete access to each channel, together with Perian, ClipWrap can be set to create multiple channel LPCM which can be independently managed in the editor. The video will still not be transcoded, and therefore there will be no video generation loss or expansion in file size in the video portion (only in the audio portion).
- If available in the original file, matching timecode will be included in the copy created by ClipWrap.
- As a setting in the preferences, ClipWrap can preserve the source file creation time, which is great for FCP X’s events, and I really can’t imagine when anyone would not want to have this option active.
Thankfully, ClipWrap has a very simple user interface, and as a result, there is likely nothing that you could possibly do wrong when using it. Perhaps the most important thing to do when first opening the ClipWrap program is to set the Movie Destination folder, which is easily seen and adjusted at the very bottom of the ClipWrap window. Next, it would be good to set the preferences to your taste, most of which I covered above. To select the raw footage, you can use any of four methods:
- Drag from the Finder.
- Click on the + symbol on screen.
- Use your keyboard to press Command together with the letter O.
- Use the menu option File>Open…
In any of the four cases, it’s best to grab the highest level AVCHD folder (not the actual MTS files) so that ClipWrap can use the metadata to treat the footage the best way possible and (if appropriate) rejoin spanned clips that were forcibly divided by the FAT32 monster in the camera.
If you have a very recent editing program that edits H.264 natively (and if your computer is also fast), then you’re probably best served by using the default mode of Rewrap (don’t alter video samples), since this will be much faster and the video file sizes won’t multiply. If your editing program is not good at editing H.264 natively (i.e. FCP 7 or earlier) or if you have an older, slower editing system, them you may want to use one of the offered transcode options on the list, which currently include most flavors of Apple ProRes (all but ProRes4444), and Avid’s DNxHD (if installed). This is a good time to mention that at least as of publication time, ClipWrap will even run on PowerPC Macs, so ClipWrap can help you stretch the life of both your older hardware and older software while making them compatible with AVCHD, although it will take much longer and take much more drive space if you transcode.
Apple App Store? May you install ClipWrap on all of your Macs?
The current version of ClipWrap as of the publication date of this article is 2.4.6 and is sold exclusively via the Divergent Media website. However, the EULA (End User License Agreement) does allow you to install it on multiple Macs for the price of one. It is possible that a future version of ClipWrap may be sold via the Apple App Store, but the current one wouldn’t be allowed there due to some QuickTime tricks it does.
Windows version of ClipWrap?
Sorry for our Windows readers, but Colin tells me that there isn’t any plan to offer ClipWrap for Windows. If any reader knows of a similar app for Windows, please write it in the comments below. In the meantime, you should know that you can use ClipWrap on a Mac to rewrap (or transcode files to Apple ProRes or Avid DNxHD), all of which can be read on a Windows machine.
Why did I say “at least in the short term”?
I wrote “at least in the short term” for the following reasons:
- Some of you may be purchasing an external recorder, some of which fortunately use UDF formatting (Sound Devices PIX 220 and PIX 240) or HFS formatting (AJA KiPro and KiPro Mini) and fortunately have reverse telecine on board to remove all types of pulldown (2:2 and 2:3).
- Some or all of the pro AVCHD camera manufacturers may take my suggestions indicated in my open letter in part 6 of the PsF’s missing workflow series.
- Eventually, Adobe, Apple, and Avid may discover the hidden, non-standard flags that Colin discovered in the malignant PsF footage, and then add direct support to recognize them directly as progressive. However, no one outside those companies knows when that might happen, and the upgrade cost from your current editor might be much more expensive than the price of ClipWrap.
In the meantime, if you are a Mac editor who receives AVCHD footage (especially with malignant PsF or AC-3 audio), or a Mac editor with an older system (with older software and/or a PowerPC Mac) and are receiving AVCHD footage of any type, ClipWrap will likely be your inexpensive bridge over troubled waters.
Upcoming articles in the PsF’s missing workflow series
Upcoming articles in this PsF’s missing workflow series will reveal the different workarounds with several other software programs, and my conclusions about this situation. To be sure you don’t miss any articles, sign up for my mailing list here.
Prior sections of the PsF’s missing workflow series
- PsF's missing workflow, Part 1: BENIGN PsF versus MALIGNANT PsF from October 23, 2011
- PsF's missing workflow, Part 2: the Canon XA10 camera from October 26, 2011
- PsF's missing workflow, Part 3: Sony's AVCHD & NXCAM cameras from October 31, 2011
- PsF’s missing workflow, Part 4: file-based HD video recorders from November 10, 2011
- PsF's missing workflow, Part 5: Átomos hires a stripper! from November 11, 2011
- PsF’s missing workflow, Part 6: T©pper asks the camera manufacturers… (An open letter to all professional AVCHD camera manufacturers) from November 14, 2011
- PsF's missing workflow, Part 7: Adobe Premiere Pro CS5.5.x from November 28, 2011
Allan Tépper's books, consulting, articles, seminars, and audio programs
Contact Allan T©pper for consulting, or find a full listing of his books, articles and upcoming seminars and webinars at AllanTepper.com. Listen to his TecnoTur program, which is now available both in Castilian and in English, free of charge. Search for TecnoTur in iTunes or visit TecnoTur.us for more information.
Disclosure, to comply with the FTC's rules
None of the manufacturers listed in this article is paying Allan T©pper or TecnoTur LLC specifically to write this article. Some of the manufacturers listed above have contracted T©pper and/or TecnoTur LLC to carry out consulting and/or translations/localizations/transcreations. Many of the manufacturers listed above have sent Allan T©pper review units, Adobe gave him an NFR (not for resale) license of the Adobe programs described, and Divergent Media gave him an NFR of ClipWrap. So far, none of the manufacturers listed above is/are sponsors of the TecnoTur programs, although they are welcome to do so, and some are, may be (or may have been) sponsors of ProVideo Coalition magazine. Some links to third parties listed in this article and/or on this web page may indirectly benefit TecnoTur LLC via affiliate programs.