Spatial Memories

February 11, 2024

Friday (the 2nd) was launch day for Apple Vision Pro. Two days later, I found myself on a newly-installed L-shaped bench in the Apple Store, sitting next to Nick.

While Nick explained how the demo would work, a different employee silently placed a Vision Pro on wooden platter between us.

(The in-store experience is something to behold on its own – but for that I’d recommend Michael Steeber’s article here. He’s been obsessively covering Apple Stores for years, and I love his work!)

The most stunning moment of the demo came not from the Hollywood-style sizzle-reel of sports and rhinos coming at you. It didn’t come from pretending to be on the moon.

That’s fun – but doesn’t have staying power for me personally.

It instead came from a home video of a young girl’s birthday party, where she blows out the candles on her cake, laughs, and falls back into her mother’s arms.

Apple is calling it “spatial video” – which certainly sounds cooler than “3D” or “stereoscopic” video.

Upon starting the video, the Vision Pro dimmed the my environment. The window itself had no clear borders. Instead, it was blurred, giving the effect of looking into a square-ish portal.

Then, I was there. The girl, the cake, the candles. Right-in-freaking-front-of-me.

Yes, you can make fun of me – I started tearing up in an Apple Store. But that’s the thing – I wasn’t in the Apple Store at all.

The video playing in front of me was contained within a floating window. The Hollywood-style sizzle-reel played later on, encapsulated my entire field of view. Technically, the sizzle-reel should have been more “immersive”.

But the girl blowing out the birthday candles was more intense. Because it was personal.

My wife, who also demoed the Vision Pro, said it was akin to a Pensieve in Harry Potter – a device that lets you peer into another’s memories.

At this point, I paused the demo and looked over to Nick. Yesterday they had to stop a demo because a man starting bawling at the video, he told me. The video reminded the man too much of his own kid’s birthdays.

Nick shared that he wished for these types of videos of his grandfather, who had passed a few years prior.

If I bought a Vision Pro today – this would be the reason: to record spatial videos of my wife and I having fun in Austin, my mom playing with my niece, or my dad telling stories.

I couldn’t imagine having spatial videos of kids growing up – but now that possibility is here.

In the days following the demo, I’m left with two thoughts. The first is a technical note.

The video itself – while impactful, was grainy and somewhat low resolution. The limitations of iPhone-size camera sensors become apparent when viewing photos/videos larger-than-live in the Vision Pro, no matter how much computational photography Apple throws at it.

It makes me yearn for a product that doesn’t exist yet – a consumer camera that can capture spatial video!

How cool would this be? If someone made this – I would unload a stupid amount of money for it.

The second thought is more hazy.

Suppose I find myself toward the end of life, with 40 years worth of spatial videos collected. (I’m 27, to clarify.) I have vivid, repayable memories of parents long lost, the full spectrum of my kids’ lives, and ever trip I’ve ever taken.

Is this too much? Even if I could bring myself to watch them – is this psychologically healthy?

My parents have a difficult enough time watching the sparse number VHS tapes from my childhood. What about an IMAX-sized spatial memory?

Maybe flat photos hold a sweet spot between “too real” and “not real enough” – they allow us to remember, but also move on. You cannot live in a photo the same way you can live in a spatial memory.

It might take time to work these things out as a society. But regardless, it’s happening. The technology is here, the Vision Pro is here, and it is thrilling!

Update on 6/10/2024: It looks like the idea above is coming to life!