Over the years, I have seen a lot of folklore and bad math employed to determine how to work with non-square pixels, resulting in a plethora of incorrect working practices. Therefore, in this article I’m going to spend a lot of time laying out the historical and mathematical basis for where these numbers came from. Hopefully this will provide you with a solid foundation on which you can build a new set of working practices.
PARs in CS4 = FUD
With the release of their Creative Suite 4 (aka CS4), Adobe introduced across their product line a “new” set of pixel aspect ratios for standard definition non-square-pixel media and compositions. This has caused widespread fear, uncertainty, and doubt, as many users suspect their old workflows have now flown out the window.
Relax. For one, these pixel aspect ratios are not new – they’ve been known for roughly a decade. They are also more correct than the ones everyone has been using to date. Finally, the changes required in workflows are nowhere near as drastic – nor affect as many users – as you might think. They just require a couple of decisions.
(If you’re more of an artist than an engineer, you can skip ahead a couple of pages – although you’ll miss out on some wonderfully geeky party conversation points, along with a deeper understanding of how to navigate the non square pixel minefield.)
From Analog to Digital
In the beginning, video and videotape were both analog: a series of voltages and blips that described not just the picture, but also all of the timing information necessary to determine where each field – and each horizontal line in that field – started and stopped. These include “blanking intervals” to cover when the electron beam painting the picture on an old CRT reset itself from the right edge of the screen back to the left as well as from the bottom of the screen back up to the top. If you’re interested in the geeky details, National Instruments has a good description here; the illustration below is a sample from their site:
An analog video signal contains picture information (the squiggly lines) plus timing information (the pulses). Image from the National Instruments Developer Zone.
The ITU-R BT.601 specification defines how this analog signal should be encoded digitally, and became the stone tablet for how standard definition digital video would work from then forward. The first digital video decks – D1 (component video) and D2 (composite video) – followed this spec, making it easy for them to emulate an analog video deck. This is why standard definition digital video is often known as “601” or “D1” format video.
The Origins of Madness
To understand the logic behind the 601 specification, it is important to know two things about product design engineers:
- Digital circuits are easier to design if they work like gears meshing together. It’s hard to design a circuit to, say, divide by 29.97; believe it or not, it’s easier to design a circuit that multiplies by 30,000 and then divides by 1001 (which happens to be the true frame rate of NTSC video – not 29.97). The take-away from this: Engineers prefer to define things as ratios between whole number, rather than as decimal values.
- Whenever you can use common parts across multiple devices, you can save money. This is even a concern with professional tape decks carrying five digit price tags. So if you can find a way to use the same part in more than one device, you go for it, even if it means standing on one foot and rubbing your stomach for what may otherwise seem like no apparent reason.
These two principles heavily shaped the 601 specification. To wit, it was decided to try to use as many common numbers (and therefore, parts) between NTSC and PAL as possible. For example, both formats employ a master clock of 13.5 MHz; everything that is needed is derived from this number. Also, both NTSC and PAL “D1” formats ended up with 720 pixels (samples of the master clock) per horizontal line. So how did we get from that starting point to that ending point?
A frame of NTSC video – when you look at the entire analog signal of image plus blanking intervals – contains 525 lines per frame (not 480 or 486), running at a frame rate of 30,000/1001 frames per second. Doing the math, a 13.5 MHz master clock divided by 525 lines divided by (30,000/1001) = 858 digital samples per line of video.
Similarly, PAL video contains 625 lines, running at a frame rate of 25 fps. 13.5 MHz divided by 625 divided by 25 = 864 samples per line of video.
But doesn’t D1 NTSC and PAL video have 720 samples per line, not 858 or 864? Yes – but “720” is just the image. The rest is the blanking interval and timing information between each line: namely, 138 samples in NTSC (720 + 138 = 858), and 144 samples in PAL (720 + 144 = 864). The following figure – which is straight from the 601 specification – lays it out:
Another way of visualizing the above diagram is like this:
How a video frame is stored on analog and 601-compliant digital videotape. The two squished images are the two fields of a frame (each consisting of alternating lines of the final frame); the black area is the blanking interval during which the electron beam in old-fashioned CRTs and television sets is traveling from right to left and from the bottom back to the top.
next page: deriving the pixel aspect ratios
The Birth of Non-Square Pixels
At this point, I imagine the engineers were quite proud of themselves: Despite having to design for two different video standards with different frame rates, they managed to use a common master clock, and end up with a common image width. And of course, standard definition NTSC and PAL have the same image aspect ratio: 4:3 (we’ll deal with widescreen later).
Just one problem: NTSC and PAL frames have different heights – 486 lines versus 576 lines. How can we use the same number of horizontal pixels for both, and still end up with the same image aspect ratio? The answer: fudge how wide the pixels are drawn. NTSC has fewer lines than PAL; therefore, their pixels will need to be drawn skinnier than PAL in order to end up with the same image aspect ratio. And thus, non-square pixels were born.
What? You’re not cheering?
Of course, you were never supposed to see any of these machinations; it was all supposed to happen underneath the hood. Everything was corrected for by the time it was displayed on analog video (for example, NTSC pixels are squished horizontally at the digital to analog conversion stage). But when we started capturing video into the computer – using the 601 spec, of course – the implications of this tortured math became revealed: A digital NTSC frame looked fat on a square pixel computer monitor (which didn’t know it was supposed to be drawn skinny); a PAL frame looked skinny (the computer didn’t know it was supposed to be drawn fat):
Normally, circular objects are supposed to be drawn as circles (left). But when non-square pixel content is displayed on a computer, NTSC frames look fat (lower left) and PAL frames look skinny (below).
A Honest Mistake
For example, it’s perfectly reasonable (although ultimately wrong) to use the following logic:
- the frame’s aspect ratio is 4:3
- an NTSC frame is 486 lines high
- 486 x 4 / 3 = 648 pixels wide for a square pixel frame
- 648 square pixels / 720 non-square pixels = 9/10
- therefore, the pixel aspect ratio (PAR) for an NTSC pixel must be 9/10
It’s also perfectly reasonable to follow the same logic chain for PAL: 576 lines x (4/3) = 768 square pixels across; 768 / 720 = 48/45 for PAL’s pixel aspect ratio.
Too bad that the people who used that logic when designing desktop digital video gear didn’t have all of the facts.
Production Aperture vs. Clean Aperture
There’s one flaw in the above logic: Those 720 pixels contain more than the entire image we’re supposed to see. The original engineers knew that in video, sometimes the details along the left and right edge of the frame can get altered – such as by electronic “ringing” (overshoot and undershoot of the voltage level which describes the corresponding luminance and color) induced by substandard analog circuitry. So they built a safety margin into the spec, expecting the damaged extra pixels on the left and right would be cropped off later.
Those 720 pixels are know as the Production Aperture – in other words, every pixel that was supposed to move down the production chain. However, inside of those 720 pixels is a smaller Clean Aperture, which contains the “actual” image for display purposes. When you calculate how big the final image is, you’re supposed to use the Clean Aperture.
But early desktop digital video engineers didn’t know that, and used the Production Aperture instead. The result is a slight math error in the resulting pixel aspect ratios. Oops…
The entire captured image is considered the Production Aperture; the thin vertical lines along the left and right edges indicate the cropping that is supposed to take place to create the Clean Aperture – the final “real” image.
The “Real” PARs
So, what are the correct PARs, taking into account the Clean Aperture rather than the Production Aperture? Honestly, the history is murky, and there has been numerous false starts (click here if you really want to know the gory details), but about a decade ago, consensus among video engineers settled around the following PARs which were derived from the clock frequencies used in digital video capture cards and the such:
- NTSC 4×3: 10/11 (not 9/10)
- NTSC 16×9: 40/33
- PAL 4×3: 59/54
- PAL 16×9: 119/81
Just how large is the difference between the old and new PARs in the real world? The two images below show the difference in how a perfectly round wheel would ultimately be displayed on a properly calibrated television:
The old PARs were used to correct the DV footage on the left; the new PARs were used for the footage on the right. Can’t tell the difference? Neither can your viewers, frankly. But if you cut these two images together, you will indeed see a “pop” in size. Those with really sharp eyes might have noticed that the wheel on the right – from CS4 – comes ever so slightly closer to the left and right edges of the square box than the image on the left, from CS3.
next page: Clean Aperture and square pixel sizes
Clean Aperture Sizes
Now that we know the correct pixel aspect ratios, we can use them to determine what the Clean Aperture is for each frame size. Keep in mind that the Production Aperture sizes – the pixels you actually need to push through your production chain – have not changed; they’re still 720×486, 720×480, or 720×576. What has changed is that we’re now supposed to be aware of the Clean Aperture, which is the actual image we need to worry about presenting to the viewer.
When calculating these sizes (as well as the square pixel artwork sizes below), one more idiosyncrasy needs to be kept in mind: even numbers rule. Whenever possible, you want to either pad or crop by pairs of pixels or lines. If you have an odd number, and then divide it by 2 (for example, trying to center an image), you end up with a half pixel – which means additional antialiasing (and in turn, softening) will take place. Cutting an odd number of lines vertically also flips field order, which is usually a no-no. So the math often gets adjusted by the desire to keep pairs of pixels around the edges.
For NTSC, 486 lines x (4/3) x (11/10) = 712.8 pixels across for its Clean Aperture size. Fractional numbers are even worse than odd numbers, so this gets rounded down to 712, yielding a clean ap size of 712×486.
(Anyone remember early NTSC DV cameras that had 4 pixel wide black bars running down the left and right edges? Now you know where they came from: The difference between Production and Clean Aperture. The manufacturers probably figured that being digital, they no longer had to worry about edge ringing artifacts and the such. Plus, it no doubt saved on bandwidth…)
For PAL, 576 lines x (4/3) x (54/59)= 702.9. Using the same rounding logic as above, PAL’s Clean Aperture size is 702×576.
To summarize:
NTSC: Production Aperture = 720×486; Clean Aperture = 712×486
PAL: Production Aperture = 720×576; Clean Aperture = 702×576
(Yes, the difference between Clean and Production Aperture also affects where the safe area guides should be placed. I discuss that here.)
Square Pixel Sizes
Once we have the Clean Aperture, we can determine the correct square pixel dimensions to create artwork in applications such as Photoshop, Flash, and Illustrator. And that’s what the new pixel aspect ratios primarily affected: The recommended square pixel sizes for standard definition video graphics, with some slight collateral damage on how you move content between standard and high definition (more about that on the last page).
NTSC 4:3
First thinking of the 4:3 Clean Aperture image, 712.8 x (3/4) = 534.6 lines for the height, rounding to 534. So if you wanted to create an image in Photoshop etc. that was just The Image – and nothing but The Image – you would create at a size of 712×534.
However, if you scaled the 534 horizontal dimension down to fit the 486 lines of an NTSC D1 composition (which is based on Production Aperture sizes, not Clean Aperture), the image would not fill out the width of the composition. You can either live with the slim empty bars down the side (usually, they’ll be cropped by the TV’s bezel, although 4:3 content “pillarboxed” into the center of a 16:9 screen will re-expose them), cover for them in some manner such as filling your background with black, or – the preferred solution – create a Production Aperture square pixel image of 720×534 (the new square pixel preset in CS4 – not 720×540). Just keep in mind that some of those pixels down the left and right are excess.
The new NTSC 4:3 square pixel size is 720×534, not 720×540.
NTSC 16:9
The logic is similar: For a Clean Aperture image, 712.8 x (40/33) = 864 pixels wide. You can also get there by going 486 x (16/9) = 864; both yield a Clean Aperture widescreen square pixel size of 864×486. This indeed is the size many used previously (by the way, there was no need to use 960×540; it just wastes pixels), and you can still use it now – just keep in mind that it “only” fills the Clean Aperture, not the full Production Aperture.
If you hate empty pixels, for Production Aperture, 720 x (40/33) = 872.7 pixels. Rounding down to an even number yields 872×486 as the square pixel size to create widescreen NTSC artwork. Just again: Remember that 4 pixels on each side are excess.
Creating a new NTSC widescreen square pixel document in Photoshop CS4.
PAL 4:3
Starting again with the Clean Aperture size, 576 lines x (4/3) = 768 lines, yielding a 768×576 square pixel image – a familiar size from the past.
However, the PAL square pixel Production Aperture calculation takes a slight left turn (just when you thought you were getting the hang of this…). 720 x (59/54) = 786.7. You would think you would just round it down to 786 pixels wide. However, 786 (prod ap) – 768 (clean ap) = 18 pixels, or 9 excess pixels per side. And remember: we try to avoid odd numbers of pixels. Therefore, the BBC says instead we should round this up to 788×576 pixels, which is indeed what Adobe has done in the PAL square pixel presets in CS4.
PAL 16:9
Here’s some more familiar numbers: 576 x (16/9) = 1024, which yields a 1024×576 square pixel image – a size many used for widescreen PAL production. (Just checking our math, 702.9 x (118/91) = 1024 which proves those weird numbers actually work.) The difference is, pre-CS4 we were jamming this correct Clean Aperture size into a Production Aperture composition, resulting in pixels unintentionally getting cut off downstream.
As for the correct PAL widescreen square pixel Production Aperture size, 720 x (118/81) = 1048.9. However, throwing one last curve ball, the BBC says we should round this up to 1050×576 pixels, which is again what Adobe has done in CS4.
To summarize:
Clean Aperture Square Pixel Sizes (the old standard):
- NTSC 4:3 – 712×534 (you could also use 720×540 and scale down by slightly more to fit)
- NTSC 16:9 – 864×486
- PAL 4:3 – 768×576
- PAL 16:9 – 1024×576
Production Aperture Square Pixel Sizes (used in CS4; safest for production):
- NTSC 4:3 – 720×534
- NTSC 16:9 – 872×486
- PAL 4:3 – 788×576
- PAL 16:9 – 1050×576
480 lines: Common Fiction; Uncommon Fact
More than one user has looked at DV’s 720×480 frame size (compared to D1’s 720×486), and started calculating a whole new set of numbers for square pixel artwork based on 480 being the height. As a result, they often come up with a new pixel aspect ratio for DV footage. This falls into the flat earth category: nice try, but wrong. In NTSC, DV is just D1 with 6 lines cropped off the height (in most cases, 4 from the top and 2 from the bottom) – they have the same pixel aspect ratio, and should otherwise be treated the same.
However, there is another 480 line format. The Advanced Televisions Systems Committee (ATSC for short) – the people who brought you the high def specs – also defined a 704×480 standard definition format. This format has the same pixel aspect ratio as NTSC D1 and DV (10/11 for 4:3 images; 40/33 for widescreen), and its Clean and Production Aperture sizes are the same (it’s strictly digital – no need to compensate for edge artifacts introduced by analog processing). I have seen this frame size pop up in consumer DV recorders that allow their images to be transferred to and from computers, and have heard of its use by other MPEG streams. Although there is a chance that you might not ever encounter this format, now you know how to deal with it if you do: Just use the same PAR as for normal non-square NTSC footage.
next page: what does this all mean during a job?
Working Practices
- If you’re working strictly in high definition, relax – nothing has changed.
- Standard definition comps and sequences have not changed in size; video captures will be the same size too – so no new numbers to learn there either. (The only thing that has changed is that we now know some of the pixels are merely padding – not active image. We still get all of the pixels to play with.)
- Non square pixel footage will round trip in and out of CS4 applications without change; there is no worry about the result “no longer working” in non-CS4 applications. Your video captures and old renders will also still fill out comps in AE CS4 and sequences in Premiere CS4 without issue – so again, relax.
- If you are creating square pixel artwork destined for a non square pixel standard definition project in a Creative Suite 4 application, you need to make a decision: create at Clean Aperture sizes (knowing all of the pixels you see will be considered part of the final image), or create at the new Production Aperture sizes (knowing that a few pixels along the left and right are just padding – not “real” image). I’d say do the latter, as it will cause less confusion, cover for more cases (such as the left and right edges of 4:3 content being revealed on a 16:9 screen or video projector, in web video, etc.) and require fewer additional decisions downstream (such as how to pad out the difference between Clean and Production Aperture). Note that all of the CS4 presets for square pixel artwork use the Production Aperture size.
- If you are handed square pixel artwork created at the old sizes, you need to make a decision: do you want to scale it to fit the Clean Aperture, or the Production Aperture? Scaling to the clean ap size means scaling it so that the artwork’s height fits the comp or sequence’s height, at the cost of some blank pixels to the left and right. Although there is a certain purity of purpose in aiming just for the Clean Aperture size (after all, that was probably what the image’s original creator really intended), as noted before you can’t rely on the left and right edges being cropped upon viewing – so you need to come up with a strategy for covering those pixels between Clean and Production aperture. Therefore, it is probably better that you scale the artwork so that it fills the width of the target comp or sequence, and live with cropping a few pixels off the top and bottom. (Don’t do a “scale to fit” – this will result in the height and width being scaled differently, re-introducing a slight distortion into the final image!)
The biggest decision comes when sizing legacy square pixel assets into new CS4 compositions or sequences. If you scale to match the height, you will see the entire image, but there will be some blank bars on the left and right that need to be padded out (above). If you scale to match the width, the entire Production Aperture frame will be covered, but a few pixels will be sliced off the top and bottom of your image (below).
I mentioned earlier that there are some curve balls involved when using high def footage. In HD, the Production and Clean Apertures are the same: There are no extra pixels to play with or worry about, unlike the standard definition case. If you are fitting an HD image into a widescreen SD frame or vice versa, you have the exact same issues as when fitting legacy square pixel artwork into an SD video frame: do you pad the sides, or crop the top and bottom?
If you scale a 1920×1080 HD image by 45.0% to fit into an SD widescreen comp in CS4, you are scaling to fit the clean ap of the HD image to the clean ap of the SD image – with the consequence of blank pixels on the left and right, as shown above. Scale by 45.42%, and you’ll lose some pixels off the top and bottom, but have no blank areas to explain to someone less patient (and informed) than you.
It’s Complicated, But It’s Not Voodoo
Yes, that’s a lot to think about, but at the end of the day the changes are far smaller than many users assumed when they heard about CS4’s new PARs. Most of the time, you won’t notice any changes, and your work will merely be more accurate now. Make some decisions now on how to make square pixel artwork and how to handle legacy square pixel content in a non-square comp or timeline (in both cases, I’d work at Production Aperture size, and live with a few pixels getting cropped off), stick with them, and get on with your life. If a special case comes up, at least now you know the actual underpinnings and real math to help you determine what the best solution will be.
If this is still too much to digest, but you have to use After Effects CS4, I wrote a quick note on how to use the old aspect ratios on old projects in AE CS4. But the sooner you can adapt to the right way of doing things, the better. After all, it’s only been a decade since the correct ratios were agreed upon in the first place! I wouldn’t be surprised if other companies eventually followed Adobe’s lead and started using the correct values in their own applications.
-
Further References
Here is a series of links you can follow if you want to study these technical issues in more detail:
- LurkerTech – a good all-around reference site on technical video issues (DVMP is a similar site with a more PAL-centric view)IceFloe Dispatch 19 – Apple’s Rosetta Stone on how to encode digital video
- The BBC’s Guide to Picture Size for delivering content
- The ATSC spec for digital television in North America
- The University of California Berkeley has a fairly recent copy of the hoary old ITU-R BT.601 specification online. Grab it while you can; this is the document from which all of this madness derives
The content contained in the CMG Blogs and CMG Keyframes posts on ProVideoCoalition are copyright Crish Design, except where otherwise attributed.