The technology in Apple’s Vision Pro AR headset: M2 chip, 4K displays

Apple packs a lot of hardware into its new Vision Pro headphones, including high-definition optics, more than a dozen sensors, and a pair of Apple-designed processors. Here’s a look at the technology that makes the new augmented reality device work.

The Vision Pro’s operating system software, called VisionOS, runs on the same M2 processor that Apple uses in its Mac computers, the company said at its WWDC developer conference on Monday. But to handle all the sensor data, Apple designed a companion chip, the R1.

“It processes input from 12 cameras, five sensors and six microphones,” said Mike Rockwell, head of Apple’s technology development group. A control that quickly reduces the delay between head movement and the corresponding change in the display’s field of view to just 12 milliseconds, about one-eighth the duration of a blink. Low display latency is essential to avoid the nausea that some VR and AR headsets suffer from.

Premium electronics come at a price. The Vision Pro will cost $3,499 (about £2,815 or AU$5,290 converted) when it ships in early 2024, Apple said.

But with the Vision Pro, Apple is betting that a premium experience is necessary to overcome the hurdles that have kept VR and AR headsets out of the mainstream until now. Budget headsets from Meta and other competitors have so far failed to attract many people to promises of virtual and augmented reality such as immersive gaming and video, web browsing and word processing with giant virtual computer screens and video conferencing with life-sized satellites.

Watch this: Everything Apple announced at WWDC 2023

11:44 a.m

However, the price tag won’t get you everything. Although most of the headphones’ electronics are built-in, they are connected by a cable to an external battery.

Vision Pro optics and displays

One major component of the experience is the display. VR headsets with a narrow field of view and obviously pixelated images are limited, but Apple is trying to offer something more immersive.

The Vision Pro has two postage stamp-sized displays, one for each eye. But three lenses stretch them to cover a wider field of view while maintaining sharpness and color.

“This allows video to be rendered in true 4K resolution with wide color and high dynamic range, all on a massive scale,” Rockwell said. “Fine text looks super sharp from any angle.”

Reality Pro has a conventional camera that will allow you to take pictures. But it also lets others track what’s around you for augmented reality views of your home, office or airplane seat. And downward-facing cameras track your hands for finger gestures that let you select objects and press buttons.

To understand which parts of the screen you’re focusing on, eye-tracking hardware in the headset follows your gaze, illuminating your pupils with infrared light. Eye tracking information is not shared with website or app developers who value such information as it may reveal what is most interesting or important on the screen.

Although the Vision Pro is opaque, it has a separate outward-facing display that shows your eyes to people looking at you when you’re wearing the headset. A feature called EyeSight can also show a bit about what you’re doing, adding a blue tint when you’re using an app or flashing white when you’re taking a photo.

It’s all very complicated, very expensive, and unlikely to sell in the volume of smartwatches, much less smartphones. But in the electronics business, costs often decrease as technological expertise and production volume increase. Apple CEO Tim Cook called Vision Pro “the beginning of a journey.” Perhaps an eventual Apple Vision non-Pro or Apple Vision 2 Pro will appeal to more buyers.