Google Photos’ new Auto frame lets you recompose photos after the fact

We’ve all been there. You scroll through your camera roll and find a shot that’s almost perfect — the smile is right, the lighting is good, but the angle is slightly off. Maybe the wide-angle lens made your face look weird, or you wish you had positioned the camera just a bit lower. Classic editing tools can’t fix this. Cropping just makes things tighter. Zooming doesn’t change the parallax. You’re stuck with the perspective you chose in that split second.

Google is trying to change that with a new feature rolling out to Google Photos called Auto frame. It’s not just another crop tool. Instead, it treats your 2D photo as a 3D scene, figures out where the camera was in that space, and lets you move it — generating whatever content was hidden behind the original viewpoint.

How it actually works

The approach is surprisingly straightforward for something that sounds like magic. It happens in two stages.

First, Google’s model estimates a 3D point map from the original image. For every pixel, it figures out where that point sits in 3D space. They specifically tuned this for human bodies and faces to avoid the kind of reconstruction artifacts that make people look like wax figures. It also guesses the original focal length.

Second, they use classic 3D rendering to project that point map from a new virtual camera position. But here’s the problem: when you move the camera, you reveal stuff that was never in the original frame. The point map has holes. So they feed this incomplete render into a generative latent diffusion model trained specifically to fill in those gaps. The model was trained on pairs of images with known camera parameters — it learned to reconstruct one image from the 3D projection of another.

At inference time, they use classifier guidance with regional scaling to keep things coherent. The result is a new perspective that looks like it could have been the original shot.

What this means in practice

I’ve been playing with the Auto frame feature for a few days now. The most impressive results are with portraits and selfies. You can shift the angle slightly to get a more flattering view without the awkward wide-angle distortion. Group photos benefit too — the feature can adjust the perspective so everyone’s face looks more natural.

But it’s not perfect. The generative filling can get weird with complex backgrounds or when there’s significant occlusion. If someone’s arm was blocking a chunk of the background, the model sometimes hallucinates unconvincing details. Also, the feature works best with subtle adjustments. Push the camera too far and the artifacts become obvious.

Google is positioning this as a fix for “almost perfect” shots, and that’s exactly what it is. It won’t turn a bad photo into a great one, but it can salvage those moments where the timing was right and the angle was wrong.

The bigger picture

This is genuinely interesting from a technical standpoint. Most generative image editing tools work in 2D — they inpaint or outpaint based on pixel-level understanding. Google’s approach explicitly models the 3D geometry of the scene, then uses generation only to fill the gaps that 3D rendering can’t handle. That’s a more principled way to do this than just asking a diffusion model to “move the camera left” and hoping for the best.

The decoupling of 3D estimation from image formation means they can manipulate camera intrinsics (focal length) and extrinsics (position and orientation) independently. That gives them control that pure 2D approaches can’t match.

I’m curious to see how this evolves. Right now it’s limited to the Auto frame feature in Google Photos, which suggests specific new camera parameters automatically. But the underlying tech could eventually let users manually adjust the viewpoint. That would be a game-changer for anyone who’s ever regretted not taking a photo from a slightly different angle.

For now, it’s a solid addition to Google Photos’ editing toolkit. Not revolutionary, but genuinely useful for those specific moments when you wish you could go back and reposition the camera. And honestly, that’s more than most AI photo features can claim.