Now that Generative AI in Photoshop has moved out of beta and into the current shipping version, here’s a quick tutorial on how to create a simple “set extension” or “scene replacement” on a video clip using that powerful feature.
To pull this off you’ll need a locked down shot (e.g. shot on a tripod) with no camera movement whatsoever. Most of the scene should also be still, except for your main subject, which needs to remain in the part of the frame that will not be replaced. These are significant restrictions, but with the right shot, the effect can be quite impressive.
All that you need to do is export a still frame of the video clip from your NLE – ideally one in which the moving subject is filling the largest area of the frame – and then open that frame in Photoshop 2024. Create a selection around the area within which the subject moves, feather it a bit for a smooth transition, invert it so that everything except for the the subject is selected, and then use the Generative Fill palette to replace your current setting with whatever you can dream up. You’ll get 3 versions to choose from, and you can run it repeatedly to get the best results. Some results can look really odd and distorted; others can look quite realistic. The key part is that the AI will often create a convincing blend between the real video and the artificially created image (remember that border cannot have any movement in it).
From there, disable the background layer and export the generated image with a transparent area for the video subject as a .png file to preserve the transparency, import that image into your NLE of choice (I’m using Final Cut Pro in this example but this process will work the same in any NLE) and composite it on top of the video clip. Instant set change! You may need to perform a little color correction if you experience a gamma shift, and of course the AI generated image doesn’t move at all so your choice of subject matter is paramount – but for the right shot it’s pretty darn cool. By the way, you could use a real photo instead of an AI generated image for the set replacement: just restrict your selection in Photoshop to the boundary between the video still frame and the new image in order to generate a smooth transition between the two. It’s all in the video above, and takes under 5 minutes – check it out.