Sony shows XR HMD for Virtual Production at CES 2024

Sony revealed at CES 2024 an immersive spatial content creation system aimed at supporting creators in sophisticated 3D content creation,  and also to be used use for film production.

Now we know the name of the fifth company using the Snapdragon XR2+ Gen 2 Platform, the latest XR processor by Qualcomm Technologies: it’s Sony! The company revealed at CES 2024 that it is developing a “Spatial Content Creation System, Equipped with High-Quality XR Head-Mounted Display and Controllers Dedicated to Interaction with 3D Objects.”

This is a headset designed with content creation in mind, and presented as part of a system, that, Sony says, “is aimed at supporting creators in sophisticated 3D content creation”, The company “plans to collaborate with developers of a variety of 3D production software, including in the entertainment and industrial design fields. At launch, Sony is exclusively partnering with Siemens, a leader in industrial technology, to introduce a new solution for immersive design and collaborative product engineering using software from the Siemens Xcelerator open digital business platform.”

The head-mounted display is equipped with large-size, high-definition 1.3-type OLED Microdisplays with 4K resolution and a wide color gamut that covers up to 96% of DCI-P3. Sony’s proprietary rendering technology enables real-time, high-definition and realistic rendering of textures of 3D objects and facial expressions of human characters.

4K OLED Microdisplays

The head-mounted display provides, Sony notes, “comfort and stability required for extended creative use, by fine-tuning the balance of the device’s center of gravity, and the design and material for pads that come into direct contact with the user’s head. Additionally, the flip-up mechanism of the display part allows creators to easily and seamlessly move between physical and virtual spaces, without having to recalibrate the device when removing and wearing the entire device.”

The device features the Snapdragon XR2+ Gen 2 Platform, the latest XR processor by Qualcomm Technologies, Inc. The processor unlocks the power of the 4K OLED Microdisplays to deliver, according to Sony, “stunning image quality, as well as provides user and space tracking for seamless XR experiences, offering creators a high-performance platform for their creative workflow.”

“We designed our Snapdragon XR2+ Gen 2 to deliver 4K display resolution, cutting-edge graphics, and unparalleled performance to enable creators to build innovative spatial content that will transform all industries for the better,” said Said Bakadir, Senior Director, Product Management of Qualcomm Technologies, Inc. “It’s exciting to see Sony enter the enterprise space with their spatial content creation system and utilize Snapdragon XR technologies to unleash more realistic, detailed, and precise Mixed Reality (MR)/VR experiences that will help developers and creators push forward an even more immersive future.”

In addition to video see-through functionality and spatial recognition with six cameras and sensors in total, Sony’s system features a ring controller that allows users to intuitively manipulate objects in virtual space, as well as a pointing controller that enables precise pointing, allowing creators to craft in virtual space with controllers and keyboards, all while wearing the head-mounted display.

An immersive tool for content creation

According to Sony, “creators will not only be able to see real-scale 3D models in an Extended Reality (XR) environment with the high-definition display, but also create and modify 3D models in it. Through integration with third-party 3D creation applications, it also supports remote, real-time review between locations, making this an intuitive and immersive tool for the entire process of spatial content creation.”

Sony has released technologies that support workflow and quality of spatial content creation and facilitate use of 3D CG in various creative fields. Mobile motion capture system “mocopi,” with small and lightweight sensors and a dedicated smartphone app, enables full-body motion tracking, using Sony’s proprietary technologies. Spatial Reality Displays provide highly realistic, three-dimensional content without the use of special glasses or Virtual Reality (VR) headsets, for spatial content creation and review process. By introducing this system and collaborating with 3D creation software developers, Sony aims to further empower spatial content creators to transcend boundaries between the physical and virtual realms for more immersive creative experiences.

Sony says that by supporting split rendering, which distributes the rendering load between computers and the head-mounted display, the system now revealed is capable of stable and high-definition rendering of large-size 3D models. The company adds that the images shown now are for illustration purposes only and may not be an exact representation of the final product.

The system will be available later in 2024. Sony and its partners will announce further details including specs, launch date and regions, pricing, sales channels, and software compatibility.