Meta’s next headset has a secret superpower

Hello, and welcome to Protocol Entertainment, your guide to the game and media industry. This Thursday we get the scoop on some of the advanced mixed reality features of Meta’s next VR headset, codenamed Cambria. Also: Sonos is launching a voice assistant and Napster is back!

Meta’s next headset has a secret superpower

Meta is gearing up to make VR a lot more real: The company today announced the general availability of its Presence Platform, which will allow developers to incorporate better hand tracking, voice interaction and video passthrough into their apps.

Passthrough essentially turns a VR headset into a mixed reality device by recording both VR and live video of the real world. For now, these efforts are very much focused on the Quest 2, whose inside-out tracking cameras only provide a faded, grayscale view of the world. But Meta’s push toward mixed reality is also clearly part of its future device roadmap, which includes a high-end VR headset codenamed Project Cambria that the company will release later this year.

I got to try a pre-release version of Cambria briefly this week after getting a more detailed mixed reality demo on a current generation Quest 2. released next week for Quest 2), but even those few minutes were enough to convince me that mixed reality on Cambria is some kind of secret superpower.

  • Cambria’s improved image sensors allow you to see the real world in color, which is a huge improvement over the Quest’s grayscale rendering. It’s still not a photo-realistic image, but it’s starting to feel a lot less jarring. Think more decent quality home video, less ‘Blair Witch Project’.
  • The new image sensors also have a much higher resolution than the Quest 2’s. Meta hasn’t shared full specs for Cambria yet, but Mark Zuckerberg told me in a post-demo chat that the sensors have three times the resolution of those used for the search 2.
  • “There’s a good roadmap for getting that even higher over time,” Zuckerberg said. “We’ll keep pushing for that.”
  • Color and higher resolution not only vastly improve the passthrough experience for Cambria wearers; it also helps the device itself make sense of the world.
  • This includes separating individual objects from others, something essential to incorporate our real environment into mixed reality experiences. While the Quest has spotted three objects stacked up in one big blob, Cambria may be able to detect clearer boundaries.
  • “Once you apply color, you can start separating things,” says Reality Labs product manager Prabhu Parthasarathy.

Cambria will be all about work usage scenarios at launch, and adding a better passthrough experience is an important part of making working in VR more natural, Zuckerberg told me.

  • Cambria is the first in a series of devices that Meta plans to sell to businesses and knowledge workers. “I think there will be a work quality device that will eventually [be] a replacement for a laptop or workstation,” says Zuckerberg.
  • By the end of the decade, such a headset could be the most important device we work with every day, and combining the real world with virtual screens and objects will be key. “You can see your desk, snap your fingers, open your screens,” Zuckerberg said.
  • In this mixed reality future, remote working will feel more like being in the same room with your colleagues, thanks to the ability to bring 3D avatars into a see-through view of your home office. “This sense of presence is something you can’t get with any other technology today,” Zuckerberg said.

Meta wants to make VR feel more natural, and color transfer is part of a broader strategy that also includes recording your hands and voice. This all depends on hardware improvements for Cambria and future headsets.

  • The Quest didn’t have a dedicated depth sensor and instead used the built-in camera sensors to calculate depth perception. “It’s kind of hacked,” Zuckerberg admitted. “On Cambria, we actually have a depth sensor.” Cambria will use an IR projector for active depth sensing.
  • The demo I got to try used hand-tracking instead of controllers, and allowed me to “catch up” with objects simply by making a grasping motion.
  • Zuckerberg said hand-tracking on Quest outperformed the company’s expectations, and Meta is doubling this input method for future headsets.
  • “With Cambria and the future devices, we now have this whole sensor architecture that’s going to be more optimized. So you just have much better hardware support for that,” Zuckerberg said.

— Janko Roettgers

Sonos launches its own voice assistant

Sonos is gearing up to tackle Alexa and Google Assistant: The company announced its own voice assistant on Wednesday. Sonos Voice Control will be available on the company’s speakers in the US early next month, with plans to launch in France later this year.

Sonos Voice Control will mainly focus on music search and playback, as well as control of Sonos speaker systems. The assistant also differs from Alexa and Google Assistant in that it never uploads audio to the cloud, but instead processes everything on the device.

  • That’s the key to winning over people who haven’t used voice assistants until now, said Greg McAllister, Sonos senior sound experience manager.
  • According to McAllister, nearly half of the Sonos speakers sold with voice functionality are not used for that purpose.
  • “When we speak to our customers, we let them know time and again that they are concerned about privacy,” he said.

Sonos also shines new light on interoperability issues that have long plagued the industry. At launch, consumers will be able to use Alexa and the Sonos Assistant on the same device and summon them with a specific wake-up word.

  • Google has long resisted this kind of voice interoperability, so people can’t use Sonos Voice Control and the Google Assistant on the same speaker.
  • At the same time, Sonos should also stay away from playing favorites with voice commands, especially since the company also has its own music services.
  • To do this, Sonos asks people to set their favorite music service in its app; if none is set, the most frequently used service is used by default and other services are treated as fallback options.
  • That’s especially noteworthy because Sonos Voice Control won’t work with every service at launch: The voice assistant can send questions to Sonos Radio, Apple Music, Amazon Music, Deezer, and Pandora, but won’t work with Spotify yet.

“We’re working to make them part of Sonos Voice Control,” promised Sonos Voice Experience Director David Leroy, without giving further details.

— Janko Roettgers

A version of this story first appeared on Protocol.com. Read it here.

A MESSAGE FROM ROKU

Hey, media professionals! Ready to dive into the entire streaming journey? A better TV storytelling experience starts with Roku Brand Studio. Chart a course and take streamers on a journey they will love today.

Learn more

In other news

The chip shortage problems in the console industry. Console makers complained about the ongoing tightness in semiconductor supplies and attributed lower-than-expected hardware sales this week to ongoing component problems. “No end in sight,” were the words of Nintendo President Shuntaro Furukawa.

EA and FIFA are splitting up. Electronic Arts and FIFA will not renew their license contract after an extension expires next year, ending their three-year gaming partnership. EA will rename the series EA Sports FC.

Ubisoft CEO tries to dispel takeover rumors. Ubisoft CEO Yves Guillemot told investors on Wednesday that his company “has everything it needs to remain independent,” following recent rumors that the company was attracting takeover bids from private equity firms.

Nintendo’s plans for a Switch successor. Nintendo thinks it will last at least a few more years in the Switch handheld, but the company said the move to next-generation hardware is a “major concern” given past mistakes around the Wii and Nintendo DS.

TelevisaUnivision acquires Spanish-language streaming service. US-based streaming service Pantaya will help Univision build its own ViX+ service, due to launch later this year.

Apple may restructure its services business. The company is considering reorganizing the company with a greater focus on streaming, according to Business Insider.

Another game studio union, in Croatia. Croatian developer Gamechuck is the latest studio to have its own union after workers signed an agreement late last month, Game Developer reported. Similar efforts to unite QA testers in the US at Activision Blizzard and EA’s BioWare have met resistance.

Google makes another tablet. The company teased a new Pixel tablet at its I/O developer conference, but people won’t be able to buy it until 2023.

Napster is back

You know what they say about cats: no, not that they are smarter than dogs, but the whole multi-life thing. That seems to extend to cat-themed startups, too, as Napster proves this week. The former file-sharing network will resurface in the coming months as a Web3-focused music service.

Previous Napster iterations featured everyone’s favorite source of MP3s; a never-launched legal music service owned by Bertelsmann; a streaming service owned by CD software maker Roxio; same service, but owned by Best Buy; a European offshoot of the American music service Rhapsody; a rebranded version of Rhapsody that was also available to US consumers; and most recently part of a UK startup looking to build immersive live music experiences.

If you do the math, that means the Web3 Napster will actually be the seventh and perhaps final iteration before the brand finally fades into obscurity. Watch out before you cross the road, little cat!

— Janko Roettgers

A MESSAGE FROM ROKU

Want great performance from your media purchase? Discover unprecedented reach with The Roku Channel and engage more than 80 million people – more streamers than on any other platform – or planet. A better TV advertising experience starts with Roku, America’s #1 TV streaming platform.

Learn more

Thoughts, questions, tips? Send them to [email protected] Enjoy your day, see you tomorrow.

An earlier version of this story said that cats have seven lives. In some cultures, that’s the number! However, it varies globally, so we updated accordingly on May 12, 2022.

Leave a Comment

Your email address will not be published.