Apple Vision Pro, the iPhone maker’s first-ever wearable headset, merges the real world and the digital and looks to usher in a new era of ‘Spatial’ computing. It’s coming soon, and it’s one of the most exciting upcoming products in the world of technology.
First revealed at WWDC 2023, Apple’s Vision Pro will take advantage of augmented reality software to let you do everything from watching TV shows to collaborating on projects. Whether you’re playing games or video calling friends, it all takes place within a headset that overlays these applications onto the real world around you.
It has the power of two 4K displays, a MacBook-grade Apple silicon processor, hand and eye tracking, as well as a Digital Crown for adjusting the level of immersion — dialing in and out the real world around you. And that’s just scratching the surface.
Apple has even created a dedicated AR division to help handle future Vision launches so it’s a core plan for the company’s long-term future.
The new headset is set to cost $3,499 and will be released in the U.S. early in 2024, with more countries to follow.
With developers already working on apps for the Vision Pro and a new alliance being formed to help drive an open standard for the creation of 3D content, the future is looking very bright for Vision Pro. Here’s a look at everything you need to know about it.
Apple Vision Pro: At A Glance
What is it?: A wearable “Spatial Computer” capable of augmented reality experiences and workflows, featuring two 4K displays, hand-tracking and an external display that allows those around you to see your face below the hardware.
How much does it cost?: $3,499
When is it coming out?: Early 2024 in the U.S., with more countries to follow
Augmented reality principles are at the heart of Vision Pro. The Apple Vision Pro uses computer-generated images as viewed through a lens, superimposed over the real world. With this, Apple says the Apple Vision Pro creates a new form of computing, called “Spatial computing,” which differs from typical mobile computing by immersing the user in the headset’s applications. Digital screens and experiences appear to form in the physical space surrounding the user, ‘through’ the headset’s lenses (which sit in front of two small screens). Apple hopes this will allow meetings to have a greater sense of presence than standard video calls, movies to be blown up to gigantic screen sizes, and workers to manipulate applications on a greater canvas than traditional hardware monitors can cater to.
Digital screens and experiences appear to form in the physical space surrounding the user.
It’s similar in principle to augmented reality apps like Pokemon Go on iPhone, but right in front of your eyes, and similar to virtual reality experiences as seen in the Meta Quest headset series, but with the emphasis on blurring the lines between the real and digital worlds.
Unlike most previous VR headsets (note that Apple refuses to align Vision Pro with the term ‘virtual reality’), the Vision Pro does not require external controllers to navigate its interface. Instead it’s managed solely by eye-tracking intent, and hand gestures. The headset’s onboard sensors are sensitive enough, and so numerous in number, that it can even track a pinch between finger and thumb with your hand resting in your lap.
Vision Pro does not require external controllers to navigate its interface. Instead it’s managed solely by eye-tracking intent, and hand gestures.
Some physical controls are retained on the headset itself though, and the most notable of these is the Digital Crown, returning from the Apple Watch. Here though, it’s used to dial in the level of immersion, letting you see more or less of the real world around you, or allowing applications to take up more of your peripheral vision as they’re overlaid onto the real world. In addition, this dial is central to Apple’s Vision Pro ‘Environments’ feature — digitized locations, ranging from forests to the surface of the moon, which can encompass the viewer filling out the real-world environment around them.
If text entry sounds difficult, that’s an issue that may be solved too. It’s also possible to look at a search field then begin dictating. Best of all? The Apple Vision Pro will identify items, text, and more through just looking at it. It’s effectively a more advanced form of Visual Lookup.
That’s just the tip of the iceberg. Read on for in-depth information on all there is to know about Vision Pro so far.
Apple Vision Pro: Hardware, Internal Specs and Design
Apple Vision Pro is powered primarily by two chips working in unison. The first is the powerful M2 chip, first seen in MacBook computers, a powerful all-purpose computational unit. A second chip inside the Vision Pro is brand new — the R1 chip (‘R’ standing for ‘Reality’) is dedicated to processing the data coming in from the Vision Pro’s many sensors, cameras, and microphones.
At least 10 external cameras are used to capture a user’s surroundings when using the Vision Pro (a pair of forward-facing cameras, four looking down, 2 TrueDepth cameras for analyzing depth, and two pointing sideways to capture the periphery of your view and surroundings), with four IR cameras dedicated to tracking in dark environments — both inside and outside the headset.
In addition, you’ve also got a LiDAR sensor for mapping distance, and accelerometers and gyroscopes to track orientation and movement.
Intriguingly, the headset will potentially have more storage than the average MacBook Pro and many of the best iPhones too. It could be as much as 1TB, but Apple has yet to confirm these details.
Let’s focus on the external design next — a unique-looking product in Apple’s line-up, elements of the AirPods Pro Max, Apple Watch and existing headset designs come together for the Vision Pro. A mixture of aluminum, plastics, and fabric mesh, there’s a 70s sci-fi vibe to it all — almost NASA-like in its mix of grays, whites, and orange highlights.
A mostly hands-free experience, Vision Pro really only has two significant hardware controls. On the right-hand top edge of the headset’s front piece is a Digital Crown, similar to what’s found in an Apple Watch. This dial can be used to turn the level of immersion up and down for the headset experiences, smoothly bringing more or less of the digital overlays on top of your real-world surroundings. The second is a dedicated camera button on the left-hand side, used for capturing 3D photos and videos.
A plastic band connects the front gasket to an adjustable fabric grip at the rear of the product, with a Fit Dial on the side to tighten the headset snugly onto the wearer’s head. The rear support is made of a stretchy breathable mesh to maximize comfort, curving to distribute weight. Similarly, the Light Seal that surrounds your forehead, eyes, and nose are well padded for maximum comfort (and available in a variety of sizes), blocking out ambient light for the best possible viewing experience of the screens inside. Those who wear prescription glasses may struggle to fit them inside Apple’s headset, but the company has partnered with eye care specialists ZEISS to provide custom prescription lens inserts that magnetically attach to the inside of the headset for a clear view. Expect this to be rather costly, though pricing isn’t currently set.
Speakers, capable of ‘Spatial Audio’ sit in pods on either side of the plastic strip connecting the front and rear parts of the headset, sitting over rather than in the ears to allow for ambient noises to be heard while in the headset.
Finally, the Apple Vision Pro will make use of an external battery pack, designed to sit in your pocket and connective over a cable to the side of the headset band via a MagSafe puck. Though that seems like a step backward for design, it is not as bad an idea as it sounds, keeping as much weight as possible away from your head. The battery is quoted as an average of two hours of use per charge, and you can plug the headset into a wall socket for continuous use, too.
Apple Vision Pro: Display and EyeSight Features
The Vision Pro uses a pair of micro-OLED ultra-high-resolution displays in its front mask, spreading 23 million pixels across them. That’s like putting a 4K TV in front of each eye. It combines this with catadioptric lenses (combining reflection and refraction of light) for sharpness and clarity.
As mentioned, prescription glasses wearers will benefit from ZEISS Optical Inserts, but those will come at an extra cost. Expect to see third-party offerings challenging the market here, as we’ve already seen with Meta’s Quest headset third-party lens inserts.
EyeSight
But there’s not just screens on the inside of the Vision Pro — there’s a wild one unlike anything you’ll have seen before on the front too.
Behind a piece of curved laminated glass is Apple’s EyeSight screen. The feature shows your eyes on the exterior of the device, based on what cameras on the inside of the headset are seeing. The display lights up when someone walks into your field of vision, revealing your “eyes” to whoever is nearby, while those around you then become visible in your AR experiences. Creepy? Early user tests show a mixture of amazement at the feature and uncanny discomfort. We will see how this plays out, but any attempt to make headset wearing a less solitary experience, or ideas to prevent having to break immersion by removing the headset, are welcome.
Apple Vision Pro: Sound and AirPods
Vision Pro features advanced Spatial Audio powered by two “AudioPods”. Each of these AudioPods has a dual-driver setup that can generate Spatial Audio based on your own head and ear geometry, delivering a simulated surround sound experience.
Apple’s latest USB-C AirPods Pro 2 are also being touted as a key accessory to pair with the Vision Pro headset. A tweaked chip design in the latest revision of the AirPods Pro 2 will allow for high-resolution audio playback, and reduced latency between what’s seen on screen and heard in-ear.
Apple Vision Pro: Hands-Free Controls and Eye Tracking
When it comes to navigating apps and interfaces with Apple Vision Pro, you are the controller. Apps and menus are navigated primarily by hand gestures, and you’re not required to wave a wand or remote to explore the headset’s software. The onboard sensors are so sensitive to hand movement in fact, that Vision Pro can recognize you tapping a finger and thumb together, even when your hands are resting in your lap. In fact, that pinching movement is the equivalent of a tap or click in the headset’s operating system, while flicks and swipes let you scroll through panes and pages.
That said, if you feel the need to hook up Bluetooth peripherals like a keyboard, trackpad, gamepad and mouse, you’ll be able to do that for those moments when a familiar typing experience or traditional game input works best.
Siri is also onboard, allowing for speech input to navigate the headset’s many features, while a Visual Search system will use image recognition to identify items in your field of view or detect and read-aloud text. So you’ll have multiple ways to get around everything Vision Pro has to offer.
Eye Tracking and Optic ID security
To ensure the accuracy of your hands-free input, eye-tracking is employed by the headset. Vision Pro has a high-speed camera and a ring of LEDs that project light onto your eyes, helping to identify where your gaze sits and to reinforce the intent of your hand gestures.
Optic ID will be used to verify log-in data in apps like Safari and to ensure payments made with Apple Pay are carried out securely.
Eye-tracking also plays a role in the headset’s security system, dubbed ‘Optic ID’. This biometric security layer takes a scan of your unique iris and uses that to confirm that only registered users can access apps and data in the headset. A ‘Guest Mode’ will also be included, allowing several users to enjoy the headset — it’s not clear yet if this is a temporary invitation, or siloed profiles with individual app data for, say, a family of users. Regardless, Optic ID will be used to verify log-in data in apps like Safari and to ensure payments made with Apple Pay are carried out securely. Apple says it does enough processing locally that websites you are browsing on the built-in browser will not be privy to information about where you are looking; the only information passed along to a given site is the click. Likewise, third-party apps get no data about what’s inside your living room.
Apple Vision Pro: Cameras and 3D Capture
Vision Pro features Apple’s first-ever 3D camera, letting you capture immersive memories in 3D with spatial audio so that you can view them again at a later date. It’s also possible to see your entire iCloud Photo Library in Vision Pro and see Panorama shots expanded as if you were standing right where they were taken.
One of the most exciting features for some users is sure to be Spatial Memories, providing you with a far more immersive experience than simply looking at a photo. For the feature, onboard cameras capture 3D video with depth, making it feel as if a recorded scene is playing out right in front of you. Those being recorded should be aware of the fact, as the front EyeSight panel will change to show a recording icon if you’re concerned about being snapped unexpectedly.
To back up this feature, the latest iPhone 15 Pro and iPhone 15 Pro Max are capable of recording Spatial Memory videos, which can then be transferred to the headset, meaning you’ll be able to capture these sorts of clips even if the Vision Pro isn’t with you.
Apple Vision Pro: What can it do?
Apple Vision Pro has a plethora of use cases, giving you infinite screen real estate. Remember — this isn’t a traditional VR headset, it’s an AR headset. So while it doesn’t support ‘room-scale’ experiences (the type that completely removes your 360-degree surroundings with digital worlds), it does allow you to have a seemingly endless amount of room for your apps and projects, spread out ‘over’ the real world around you.
Everything is built around an all-new operating system called visionOS. It builds on familiar foundations from iOS and macOS, with recognizable icons and Apple-style menu and file structures, but is designed to take advantage of an augmented reality canvas.
Expect to see visionOS versions of popular apps like Apple TV Plus, Music, Photos, Freeform, Safari, Notes, Messages, Mail, and more in the headset — as well as apps exclusive to the hardware. An App Store will let you browse the library of available Vision Pro software, as well as thousands of iPhone and iPad apps — a huge number of 2D apps will be available to be used in Vision Pro, presented as floating panes within the OS.
So, using gestures and swipes, you’ll be able to put all manner of apps and workflows around you, resizing windows to enormous sizes around you, playing back immersive Apple TV Plus videos, or making use of Continuity features with Mac applications, engaging in FaceTime video calls, and more. Here’s a run-down of the most interesting features we’ve seen so far.
Entertainment, Environments and gaming
Vision Pro looks like a stunning way to enjoy movies, TV shows, and games. It can create virtual movie theater displays of up to 100 feet wide and take advantage of advanced Spatial audio playback. It’s not just 2D content displayed at dramatic screen sizes, either — a new Godzilla show, for instance, is one particular show that will offer Spatial Video support once released to make it appear as if the titular monster star is rampaging around your living room in 3D.
Spatial computing means you can also play games on these giant virtual screens, with support for popular controllers and Apple Arcade games. Game engine pros at Unity have already been confirmed to be involved with game development so it’ll be fascinating to see what unique games emerge to take advantage of Vision Pro’s feature set.
Apple has also teamed up with Disney, bringing Disney Plus to Vision Pro at launch. While details are yet to be confirmed (we’ve only seen a conceptual sizzle real at this point), touted features include immersive settings for shows like The Mandalorian, letting you watch the show in your very own landspeeder on the sandy Star Wars planet of Tatooine. Other teased features included a virtual Mickey Mouse bounding around your home, and birds-eye views of NBA games, playing out on a tabletop in your home, like a miniature board game. Netflix fans will disappointed to learn that there won’t be a dedicated Apple Vision Pro Netflix app, however — at least at launch.
For those looking to chill out in Apple Vision Pro, the company is also introducing an application called Environments. This displays intricately-detailed recreations of places real (forests, beaches) and unreachable (the surface of the moon), and can be dialled in and out with the Digital Crown, transporting you to virtual locations around the globe from the comfort of your sofa. These can even be used alongside the 3D video playback options to further heighten immersion.
Work and productivity
While entertainment features may lure in casual users, Apple wants Vision Pro to be a productivity and enterprise tool, too. Early footage of the headset shows people working with spreadsheets, documents, and 3D modeling applications in Vision Pro for instance, and we already know apps like Final Cut Pro will offer Vision Pro support in the future with many more set to come.
A key part of this professional application bubble will be Apple’s Continuity system. The same, existing feature that lets you work seamlessly across iPhone, iPad, and Mac, sharing files, clipboards, and even peripherals across distinct devices will be in play with Vision Pro — and supercharged. For instance, the Vision Pro can recognize a MacBook Pro in front of it, use Continuity to recognize it’s a user’s system, create additional virtual displays around the physical one, and tap into iCloud cloud storage to share files across all devices. With Vision Pro, you may never need an external monitor ever again — let alone a desk to house it on.
FaceTime
FaceTime will be a huge part of the Vision Pro experience. Apple expects you to make video calls within the headset, with your chats with friends, family, and colleagues taking place in floating panes around you.
But, if you’re wearing a headset, what do those you’re calling see? A digitized version of yourself, that’s what.
Vision Pro can scan your face using the TrueDepth camera it houses, using your facial map to create a digital ‘Persona’ that can be shown to those calling you. The aim is to create a photo-real recreation of your features and expressions, reflecting your face and hand movements in real-time. With your Persona you can watch movies, browse photos, or work together with others, just like you were there with them. Kind of. There’s an uncanny quality to what has been shown so far, impressive as it is. But reports have suggested Apple is making significant leaps with the quality of these Personas in the run-up to launch.
Other features include being able to receive 3D objects in messages, letting you manipulate CAD-like attachments in 3D space.
Apple Vision Pro: Expected release date
Apple’s Vision Pro headset will initially only be available in the U.S. when it launches in “early” 2024. More countries will see a release date later in 2024.
Customers can go into an Apple Store to learn more about the headset and even personalize their fit for their own purchase, something that’s likely to be near vital given the way it works.
In the meantime, the visionOS SDK is now available to developers, allowing them to cook up new apps and experiences ahead of that launch date. It’s a reasonably long time between announcement and release (even one that hasn’t got a concrete date attached). But that makes sense, given the secretive nature of the production of the project, which now requires outside developer support to increase its attractiveness.
Apple Vision Pro: Price
Apple’s Vision Pro headset starts at $3,499, which is even more expensive than the rumored $3,000 price tag that was floating around prior to its announcement. That’s significantly pricier than even the Meta Quest Pro, and a lot more than headsets such as the Meta Quest.
Despite that, it’s still winning over the Apple faithful, with Apple Vision Pro winning ‘Most Wanted’ in the iMore Readers’ Choice Awards.
Apple’s vision of augmented reality looks more thrilling the more we learn about it. It’ll be fascinating to see what developers come up with as they work on new applications for the headset — and whether or not that price point is surmountable for those interested in Apple’s next big thing.