JVET-AO0210 AHG9: Splatting information SEI [J. Boyce, M. M. Hannuksela (Nokia)]
A splatting information SEI message is proposed, which provides support for representing a volumetric scene with a variety of splatting types, including 3D Gaussian splats. Additional splatting types supported in the proposal are 2D Gaussian splats and Beta splats. Because the technologies for volumetric scene representation are rapidly evolving, the proposed design is extensible to add support for additional splatting types and includes support for a custom type determined by the application.
Characteristics of the proposal can be summarized as follows:
- Support for multiple splatting types: 3D Gaussian splats, 2D Gaussians splats, Beta splats
- Table listing allowable attributes and attribute components for each specified splatting type
- Reserved splatting types for future extensibility
- Support for unspecified (application determined) splatting type
- Support for packing of attribute components into multiple layers and/or multiple temporal pictures
- Each picture split into a grid of same size regions, optionally separated by guard bands, with an index assigned to each region
- Per layer indication if attributes with 3 components have all 3 components placed into a single region or into separate regions
- Flag to indicate if all allowable attribute components are present
- Flag to indicate if default region packing used
- If not default packing, signal indication for each attribute if it is present, and when present, signal for each attribute or attribute component its layer index, POC, and region index
- Optional transform per attribute
- Optional scale and offset values for each attribute component
The general concept is similar as for JVET-AO0180, as both JVET-AO0180 and JVET-AO0210 were developed from JVET-AN0100. The basic approach of packing could be similar, but JVET-AO0210 tries to introduce additional flexibility for supporting more types of splats, and packing into multiple pictures.
It was commented that it might be undesirable to define too many different packing formats, as it might increase the complexity of decoder implementation.