This is a long post. TL;DR: what did Apple do to make Vision Pro’s passthrough not feel like crap despite not being depth-correct?

Depth-correct passthrough: what’s the big deal?

It’s well documented that it’s critical for video passthrough mixed reality to be “depth-correct” (AKA perspective correct) or you’ll have issues:

• Unpleasantness that persists even after removing the headset
• Can’t catch a tossed bottle
• Can’t grab things or estimate distances, feels like flailing, looks totally weird, and feels unusably bad
• Pico 4 passthrough unusable, disorienting
• “The importance of depth-correct passthrough reprojection… absolutely cannot be understated and is a make or break for general adoption of any MR device”

But perhaps the best analysis is this one by /u/kguttag. The conclusion is clear: if the passthrough is not corrected via reprojection to account for the difference between the locations of your eyeballs vs cameras looking outside (good quick explanation), you’re going to have an experience that feels weird at best and at worst unusable, disorienting, or even dangerous. You’ll have bigger problems than not being able to catch a ball.