JVET-AM0181 AHG9: Proposed new SEI message on localization and mapping [G. Teniou, S. Wenger, A. Hinds, J. Ricard (Tencent), Louis Kerofsky, Yong He, Shuai Zhao (Qualcomm)]

This document proposes a new SEI message designed to support the 3d model creation from 2d video streams. The proposed SEI message conveys essential metadata such as camera intrinsic (focal length, principal point, …) parameters and camera extrinsic (position and orientation) parameters. This metadata is crucial for algorithms that create 3D models such as 3d Gaussian Splatting (3DGS) and to assist the rendering process.

The radial distortion aspect of this SEI message is a subset of what is provided in the Lens optical correction (LOC) SEI.

It was questioned why 3dmr_camera_id is included in the intrinsic parameters.

The focal length is expressed in pixels, which should be for the capture resolution, which may not be the same as the coded picture.

This condition is problematic for future extensibility: if( 3dmr_camera_model ! = 1 )

It was suggested that gaussian splat coding may benefit from having the same information, so there should be some coordination.

Differences between JVET-AM0103 and this contribution:

• presence of the intrinsic parameters in this contribution
• the coordinate system in JVET-AM0103
• depth range
• more models for the extrinsic parameters is JVET-AM0103

There is some support for the providing a solution for this use case.

Further discussion requested after side activity to try to merge JVET-AM0103 and JVET-AM0181.

Further discussed at 1135 on 1 July 2025.

The message is renamed...