New years’ resolutions traditionally involve grand, life-changing pronouncements of such vast scope that even the friendliest onlookers tend to sip their coffee while taking time to consider a suitably diplomatic reaction. The more modest hopes discussed below are, hopefully, achievably modest, although the fact that they’re technologically fairly trivial, yet don’t already exist, might not fill even the most charitable observer with a huge amount of confidence.
Today’s sermon arises from a conversation with someone (who must, regrettably, remain anonymous) who had shot a series of small productions on a high-end cinema camera, leveraging that camera’s large format to create different effective sensor sizes for each one. It was an interesting creative approach, with different configurations taking in different degrees of the lens’s, ahem, character, different combinations of field of view, depth of field, aspect ratio and resolution, maintaining an underlying consistency while allowing each piece a degree of individuality.
It also meant taping some pieces of paper to a wall and shooting framing charts so that the editor could tell where the edge of frame was supposed to be. Much as that’s a heartwarming continuation of a near century-old craft technique, it seems a little inadequate that there is, in current practice, no widely-accepted, non-destructive way of expressing a cropped from in the files produced by cameras.
Which pixels were those, again?
We can send vast colourspaces with enormous dynamic range and gigantic resolution at huge frame rates alongside a dozen channels of mastering-quality sound, totalling anything up to a few gigabits per second, but a rectangle is somehow beyond us. Said rectangle might be defined by two sixteen-bit numbers per corner (accommodating any frame size in an image up to almost 65K square) and occupy an eighth of a kilobyte of data per frame. There’s probably more alignment padding than that in most container formats, and it’d easily fit into an SDI VANC header, but it seems curiously beyond us to make this happen.
This has only really become an issue since the performance of cameras has made it plausible to deliberately throw away pixels, but even that’s not a very new situation. Alexas right up from the original EV had bigger sensors than they commonly needed, at least for a good HD finish. Almost all modern cameras can do the same thing, and of course many of them can use that extra space for viewfinder look-around. In camera, it’s trivial to put a frame line on the image and export that to a monitor, or to have the monitor superimpose its own frame.
Embedding the data in the SDI feed would make it possible to automate that, ensure it matched the camera’s idea of the frame, and to offer one-touch selection of various viewing modes. That’s not usually possible, and framing information doesn’t generally make it to post either. When the intent is to use the full image, that’s a null issue. But it’s hardly uncommon protect much larger frames than we’re intending to use; that’s one of the more powerful capabilities of modern, high-resolution cameras, and a large part of the reason more things are now fixable in post.
Fincher is famously a fan of doing this, which is how most of that inhumanly-precise framing comes from. But even at his level, the favoured solution still involves pieces of tape on the monitor at least sometimes. This is embarrassing. We now have a system for recording the inclination and acceleration of lenses on cinema cameras (actually, we have at least two mutually-incompatible systems, but… anyway). Mostly, though, we don’t have a universally-accepted way of showing what the picture is supposed to be.
Frameline is not complicated
Like similar issues with colorimetry and LUTs, this is not a technological difficult problem. It’s even hard to see how it could be politically difficult, given it’s difficult to imagine anyone having design rights over a set of eight numbers describing a rectangle (though given the protect-the-sound-of-your-snoring permissiveness of the patent system at the moment, let’s not pray at the altar of trouble for problems we don’t want). It’s just a feat of standardisation, and like LUTs, it’s something which is only really valuable once it’s sup
ported widely enough that we can assume it’ll mostly work.
That’s a bigger ask than it should be, but still. Essentially every recording file format has ancillary space for exactly this sort of thing, and in ways that doesn’t need to break backward compatibility with everything. There’s also plenty of space for it on SDI, HDMI and DisplayPort.
It’s eight numbers, and just drawing a box on the picture is something that basically all devices can trivially do in real time. So maybe, during 2023, let’s hope that something like this can at least start becoming available. Pretty please?