A PMW-EX3, PMW-F3, and AG-AF100 lined up on a cart.
This past Monday, Art Adams and I went up to Chater Camera to have a quick look at the tonal-scale renderings of the PMW-F3 and AG-AF100, two new large-single-sensor (LSS) interchangeable-lens camcorders, and to torture them with resolution, aliasing, and IR sensitivity tests. [updated 2001-03-19: RED M vs M-X aliasing comparison.]
The Panasonic AG-AF100 is a US$5,000 micro-four-thirds LSS camera recording 1080p to SDHC cards using 24 Mbit/sec AVCHD (add $1,200 if you want a Hot Rod Cameras PL mount). Art has already shot a spot on the AF100, but we hadn’t had a chance to quantitatively evaluate the camera before.
The Sony PMW-F3 is a $16,000 APS-C / Super35mm LSS camera shooting 1080p to SxS cards recording 35 Mbit/sec XDCAM EX (PL mount included). It’s pretty much brand-new and neither of us had worked with it, though I had seen it at HPA last month.
Chater Camera, one of northern California’s top rental houses, kindly offered us access to both an AG-AF100 and a PMW-F3, and a selection of PL-mount lenses to use on them. We put a 65mm Arri Ultra Prime on the F5, and a 50mm Ultra Prime on the AF100; these gave the cameras comparable angles of view, as the APS-C sensor on the F3 is about 25% larger than the four-thirds sensor on the AF100.
At Chater, we assembled our dynamic range measurement system. A DSC Labs 102-dB 18-step (17-stop) combi on an Ambi illuminator is set up at one end of a black-lined cardboard “tunnel”, and the camera under test is set up at the other end, its lens poking through a slit in the black Duvetyne sealing the end. The range of the combi chart is so great that we have to tent off all spill and ambient illumination, lest we “fill” and lift the dark bars of the chart with stray light.
Chater’s Jay Farrington rigs a 25″ Panasonic LCD and a Leader WFM on stands beside our tunnel.
Jay Farrington and Art Adams discuss the test procedure.
We then ran through a variety of gamma curves and gain settings on both cameras, recording the total dynamic range realized in each setting and seeing how the cameras distribute tones between white and black for each one.
Art sets iris so that the brightest bar on the combi chart just starts to clip. We can then measure stops of DR by counting the distinct steps visible between clipping and black.
Art triggering record on the F3. The F3’s menus and buttons are very similar to those on an EX1 or EX3, but they’ve shifted around a bit with the different physical design of the camera.
Jay and Art view image and waveform.
I’ll let Art discuss the results in detail; we saw some interesting things. I’ll just summarize:
• Both cameras maintain fairly constant dynamic ranges through all their gamma curves; the different gammas do good jobs of affecting the overall “look” of the images while grabbing as broad a range of brightness as possible. The AF100 captured 10+ stops as far as I could tell; the F3 managed 12+ stops (the combi chart steps in 1-stop increments, so it’s hard to get a much more precise reading. In some gammas there was a hint of another stop just creeping up out of the black, but it’s hard to quantify).
• The F3 appears to have the same saturated-highlight handling problem in standard gammas as the EX1 and EX3.
• The F3 likewise shares the other EX-series cameras’ extended red response—or an excess sensitivity to infrared, depending on your viewpoint. The AF100 is much less susceptible to IR “pollution”.
Art’s black samples chart, illuminated by tungsten. AF100 on left, F3 on right.
Art had his selection of IR-cut filters handy; he’ll describe which ones he found most suitable for the F3.
Next: the resolution / aliasing torture test…
Resolution and Aliasing
I aimed both cameras at the prototype 2000 TVl/ph DSC Labs MegaTrumpet 12 chart that DSC kindly supplied me with at NAB 2008. (The chart wasn’t very evenly lit, but don’t worry about that; the unevenness doesn’t materially affect the results. Just don’t compare the charts and try to derive any useful info about tonal scale or contrast rendering.) I took representative frames into Final Cut Pro and exported stills for your viewing pleasure. Note that I didn’t tweak the cameras beforehand; the AF100 in particular shows more edge enhancement than I’d normally shoot with.
Boy, were we surprised:
AF100, 1:1 sample; 1920x1080p AVCHD transcoded to ProRes422 in FCP 7.
PMW-F3, 1:1 sample; 1920×1080 XDCAM EX HQ.
Crikey!
Remember, resolution charts are the nastiest things to aim cameras at; the vast majority of real-world subjects are far more forgiving. Compared to various versacams, these LSS camcorders aren’t so bad. But even so, these charts were a bit startling.
AF100: No Chroma Moir©, But…
The AF100 manages to slightly out-resolve the Panasonic DMC-GH1 “versacam” despite having a similar 12 Mpixel sensor (the BBC’s Alan Roberts rates it at 680 TVl/ph, whereas I see the GH1 cleanly resolving maybe 630 TVl/ph), and like the GH1 its images are admirably free of chroma moir©.
In practical shooting, the AF100’s images are just a little bit soft, and a wee bit aliasy on fine detail; nothing that makes the camera unusable in the majority of situations. Its 680 TVl/ph may sound low, but it handily exceeds the 540 TVl/ph resolution of the HXV200 and its successors, and those cameras have satisfied many owners: raw resolution ain’t everything.
Still: it’s got 12 megapixels—that’s about the same as a RED ONE—and this is the best it can do?
What about all that aliased detail out past 1000 lines? Panasonic claims that the optical low-pass filtering (OLPF) of the AF100 is optimized for 1080p, but neither the BBC’s Mr. Roberts nor I would agree wholeheartedly (yes, it’s better than the GH1, so Panasonic has done something, but it’s not that much better).
Mind you, the OLPF for a 1-chip color-filter-array camera needs to let more detail through than the recording resolution might suggest, so that the individual photosites have something to work with: passing more detail than would otherwise be required is a necessary tradeoff. I’ve shot a fair bit of video with my “GH13” hacked DMC-GH1 recently; it’s lower-res and more aliasy than the AF100 on the charts, but the only time it’s really visible in real-world shots is an occasional slight steppiness to nearly-horizontal or nearly-vertical lines. I prefer it to my Canon DSLRs for most video work; I don’t fret about the minor luma aliasing in the GH1 nearly as much as about the lurking chroma demons in the Canons. As the AF100 exceeds the GH1 in spatial rendering quality, I’d worry even less about it.
Mr. Roberts measured 25 msec of readout skew on the AF100, and that’s what I found for the GH1 in 1080p mode. Combined with spatial imaging characteristics, and the 8-bit HD-SDI output of the AF100, it really does look like the basic sensor and processing of the GH1 were borrowed (or at least used as the basis for development) for the AF100.
The AF100’s 8% boost in clean resolution over that of the GH1 and its different highlight handling and dynamic range do tend to indicate that the imaging pipeline wasn’t taken unchanged from the still camera. The GH1 does a better job of highlight handling, whereas the AF100 has a traditional video knee with a sudden onset and substantially more hue distortion in bright, saturated colors; on the other hand, the AF100 manages a couple of stops more dynamic range and offers a much cleaner image at high gains/ISOs. Nonetheless, the similarities in the two cameras’ images are striking. One could easily intercut the two cameras without seeing glaring discontinuities.
PMW-F3: Shooting Black & White… in Color!
The PMW-F3’s chroma moir© was a bit shocking, reminding me of the Canon HDSLRs (though the HDSLRs are much worse). Of special interest is that 45 degree diagonal running down from left to right: much more green/magenta aliasing than on the opposite diagonal. We tried tilting, moving, and rotating the chart to see if it was simply an unfortunate coincidence, but that tendency for “bar dexters” to go colorful persisted. I can only surmise that the demosaicing algorithm used in the F3 combines the diagonal green photosites in the Bayer pattern in a similar direction. Be mindful of scene elements on an F3 shoot that have this sort of fine left-to-right diagonal detail; they might get colorful on you.
The F3 uses a 3.36 (effective) megapixel sensor. Assuming “square pixels” on the sensor and a traditional horizontal/vertical grid (not necessarily a wise assumption, but good enough for a thought experiment!), that works out to roughly 2444×1374 photosites, or 1.27x HD resolution linearly (1.6x in terms of total area). As a general rule, one needs about 25% more photosites linearly than one expects to resolve after demosaicing, so this 27% oversupply gives Sony the theoretical margin to get full 1080p resolution out of the sensor.
The enthusiastic chroma moir© suggests a lenient OLPF to allow maximal detail, combined with a demosaic process that likewise emphasizes detail at the expense of some aliasing.
In playing with the F3 at Chater, it rendered a crisp image, but thin, dark horizontal lines (like those in Chater’s roll-up gate) had a tendency to take on a green or magenta cast, while fine repeated details and textures sometimes showed a slight color moir© or shimmer. One of the fabric patches and the black gaffer tape on Art’s IR test chart generated faint rainbow effects as the camera gently panned across them.
Again, it’s nowhere near as bad as a 5D Mk II or a 7D… but it’s not as clean as an Alexa or a RED.
Speaking of which, how do those cameras compare?
Letting the Big Dogs Come Out To Play
The Arri Alexa has a 3.5K sensor, rather higher res than the F3’s, but the Alexa also offers 2K (2048-pixel-wide) outputs in addition to 1920×1080, so that higher resolution is called for.
I happen to have an Alexa chart shot in my back pocket…
Alexa, 1:1 sample; 1920x1080p ProRes422.
Alexa is doing a much better job of alias suppression, though it seems to lose a bit of resolution in the process. In practical shots, I’ve only once seen any visible chroma moir© in Alexa footage (on thin, nearly-horizontal details, as the chart would suggest); it’s a very clean camera overall.
Then there’s this RED shot from a couple of years ago:
RED ONE “M”, 1:1 sample; REDCODE28 4096×2304 freshly transcoded to 1920x1080p ProRes422 in RCX 262.
This sample was shot at 4K and downsampled in REDCINE-X at full quality using software (not RED ROCKET’s hardware resampling). It’s the cleanest of the bunch—yet it still shows traces of chroma moir© on the test chart. In real-world shooting, though, I have yet to see any noticeable moir© in RED footage.
(Update 2011-03-19: The RED M-X sensor is substantially the same as far as spatial rendering is concerned.
I don’t have a shot of this same chart made with the RED M-X sensor handy, and our REDs are currently shrink-wrapped on pallets while our studio renovations are in progress, so I’ve taken the clips from the M vs M-X test footage I still have on my MacBook Pro and stitched together a side-by-side comparison of the resolution trumpets from the DSC CDM chart in those shots.
The active width of the chart only occupied 862 pixels wide in the M-X frame, and 870 pixels in the M frame, so the 800-line limits of the trumpets scale to about 1765 – 1780 lines in these image, as opposed to the 2000 line limit in the MegaTrumpet chart shot, so it’s not a perfect substitute for the MegaTrumpet. Still, I hope it indicates that the M- and M-X-equipped cameras are substantially the same in terms of limiting resolution and aliasing levels.)
M-X vs. M sensors, 1:1 samples; REDCODE36 3840×2160 transcoded to 1920×1080 ProRes422 using whatever version of RCX was current in early July, 2010.
Are We Doomed?
So, does that mean we should spurn any LSS camera other than the RED? Should we abandon LSS cams altogether, RED included, and go back to our comfy, familiar, nonthreatening 3-chip cameras? Far from it: resolution chart performance is only one part of the overall picture.
Sure, the RED looks best on the chart—but this picture only emerges after a laborious post-processing step: alone among these cameras, the RED is incapable of delivering a ready-to-use 1080p image in real time.
The Alexa outperforms all these cameras in terms of usable dynamic range, and has the best handheld ergonomics and the most straightforward menus of the bunch.
The Sony isn’t as chroma-clean as Alexa, but it costs a fraction of the Arri’s price, and its controls, menus, SxS cards, and codec fit seamlessly into the world of PMW-EX1s, EX3s, 320s, and 350s.
The Panasonic offers 2/3 the resolution of the others, but it’s 1/3 the cost of the Sony, 1/5th the cost of a RED, and way less than an Alexa—on a pixel-per-dollar basis, it’s a winner. And if chroma moir© is your hot button, the AF100 beats them all, hands down. The AF100 is also the only camera of the bunch that you can buy media for at Frys, Best Buy, MicroCenter, and Ritz Camera.
The winner? Different horses for different courses: we’ve got lots of choices, so we win.
LSS Requires Mental Recalibration
Our initial shock at seeing the aliasing these cameras produced is partly the result of affordable 3-chip cameras getting so danged good over the past few years. Quite frankly, we’ve been spoiled. A PMW-EX1 or an AG-HPX370 shooting 1080p will happily resolve a full 1000+ TVl/ph, and shows no detectable sensor-based chroma aliasing. Even the AG-HMC40—a mere snip at $2,000—does a downright decent job with its diminutive 1/4″ 3MOS sensors.
But there’s also a bit of apples vs. oranges going on: we’ve been looking at the three-chippers on “traditional” HDTV charts, with responses only up to 1200 TVl/ph. Let’s see what an EX1 does on the 2000-line chart:
PMW-EX1, 1:1 sample; 1920×1080 XDCAM EX HQ.
Yes, it’s free of sensor-related chroma moir©, but look at that H and V aliasing past 1500 lines. We don’t see that on the usual HDTV charts!
No, not really. I’ve been shooting with the EX1 for a couple of years, and I have yet to see anything objectionable in its spatial rendering [1]. Charts like this bring out the worst in cameras, but we don’t make our living shooting charts.
The EX1 and its other small-three-sensor (STS [2]) companions make fine images, but they lack the shallow focus beloved of many filmmakers. LSS cameras offer that tasty tiny focus, but their color-filter-array chips come with demosaicing artifacts. Horses for courses again: film isn’t video, acrylics aren’t oils, hammers aren’t screwdrivers, and LSS cameras aren’t STS cameras. Pick the tool that’s right for the job, that’s all.
(But seriously: how does Panasonic manage to completely eradicate chroma moir© from the GH1 and the AF100? Why, it goes against the very laws of man, G_d, and Bayer-pattern sensors—those Panny folks have secrets they aren’t sharing!)
[1] Actually, when pixel-peeping the original daylight frames from the Green/Magenta? article, the resolution trumpets on the DSC CDM chart show a bit of aliasing in the EX1’s rendering, while the HDW-650’s rendering is smoother, with the trumpets narrowing to a gray blur. As Sony’s Juan Martinez said at the time, that’s the difference between a good OLPF in an affordable camera like the EX1, and a great but much pricier OLPF in the high-end 650. But the differences are vanishingly small; unless you’re pixel-peeping this sort of fine repeated detail, you’re never going to notice the difference.
[2] We’re going to need this new acronym henceforth, so we can properly differentiate between LSS and STS cameras simply and succinctly.
FTC Disclaimer: No material connection exists between me and Sony, Arri, Panasonic, RED, DSC Labs, or Chater camera, except that I’ve been a happy, paying, customer of all of them. No payments, material considerations, blandishments and/or outright bribes were offered me to post this article or to mention any product in a favorable or unfavorable light.
DSC Labs gave me the prototype 2000-line chart in 2008. I’ve paid for other DSC charts, both before and since.
Special thanks are due to Chater Camera for hosting us and supplying the F3, AF100, monitoring, C-stands, and lenses. Thanks also to Art Adams for suggesting these tests (look for his article on the cameras’ tonal scales, gamma curves, and IR sensitivities, coming soon to a website near you…).