JVET-AM0212 AHG9: Gaussian splatting information SEI message [D. Wang, J. Xu, L. Zhang, Y.-K. Wang, S. Jiao (Bytedance)

JVET-AM0212 also relates to software for SEI messages

It is asserted that using exitsiting image/video codec to compress 3DGS data has been widely acknownledged. In this contribution, a Gaussian splatting SEI message is specified to support such a usage. A showcase is presented to demonstrate the usage of this Gaussian splatting information SEI message.

In -v2 of this contribution, some typos are fixed and a slide deck is attached.

16 bit data is separated into low and high bit rate ranges for coding using a 10-bit codec.

Each gaussian has 59 attributes, which are coded as image data. The pixel rate may be quite large.

Different scaling ratios may be used for different attributes.

The showcase used low delay config. It split the attribute data in 4 layers.

The quantized attributes are organized into four layers:

1. posH: 3-channel, resolution (3W)×H
2. posL: 3-channel, resolution (3W)×H
3. multiAtt: 11-channel, resolution (6W)×(2H)
4. shN: 45-channel, resolution (6W)×(8H)

Different QPs were used for the layers.

It was questioned what profile/level would be required to code 3DGS data structure as described by this SEI message.

It was questioned if low level coding tools would be beneficial for this type of video data.

A PSNR vs bitrate graph was shown. The PSNR was calculated on rendered images over 3 viewports, comparing to the rendered images without video compression.questioned if the PSNR saturates at higher bitrates.

A coordination meeting with WG4 and WG7 is planned.

If technical work on this topic were to proceed, a CTC would need to be defined.

Monochrome content is coded using luma, with mid-gray for the chroma samples.

Request that proponents upload a revision of the document that includes the picture dimensions of the 4 layers used in the showcase. The sample rate can also be provided.

It was questioned if MIV could be used for coding the 3DGS data. It is noted that have been proposals to use V3C in MPEG contributions m73255, m73256, m73257, and m73258.

It was asked if subject testing has been done, especially considering potential temporal artefacts.

See also follow-up discussion in joint meeting section 7.3.1.

SEI Software and showcases (2)

Contributions in this area were discussed during 1545–1600 on Monday 30 June 2025 (chaired by J.Boyce).

JVET-AM0212 AHG9: Gaussian splatting information SEI message D. Wang, J. Xu, L. Zhang, Y.-K. Wang, S. Jiao (Bytedance)

JVET- AM0212 also relates to identifying potential needs for new SEI messages

See notes in section 6.5.

Non-SEI HLS aspects (0)

Kept as template for future use.

Plenary meetings, joint meetings, BoG reports, and liaison communications

General

The following topics were discussed in JVET plenary Monday 30 June 1300–1400:

• Status of last meeting’s output documents
  • All WG 5 documents now available in MPEG dms and ISO documents sites
  • JVET-AM2020 still missing
  • Ballot on CD 23090-15 (not uploaded yet to dms): m73679 – Iole Moccagatta will review when available and report back.
  • Ballot on DIS 23090-16: Only editorial comments – submit DIS as final standard. Next edition including new SEI messages could be started as CD in October or at a later meeting. DoCR to accept ISO editor comments, plus recommendation to proceed to publication.
• Liaison communication – none so far
• Scheduling for the remaining week (further detail on scheduling is recorded in section 2.12)
  • CfE preparation
  • Joint review of 4.x topics
• Joint meetings
  • Monday 1700–1800 with AG 5: CfE dry-run (see notes under JVET-AM0323)
  • Tuesday 1300-1400: AG 5 & WG 4: Multi-layer test (includes LCEVC, SHVC, VVC)
  • Wednesday 1615–1645 with WG 4, WG 7: Gaussian splat coding
  • Thursday 1400–1600 with VCEG, WG 2, AG 5: Next generation video coding, CfE
• Standards progression and outputs
  • DoCR preparation 14496-10 CDAM1 (Karsten), 23090-15 3e CD (Iole), 23090-16 2e DIS (Frank)
  • White paper VSEI/VVC – to be presented in AG 3 meeting Thursday afternoon
• Review status from tracks and discussion on potential open issues

Review of documents at plenary level

List of adoptions from HLS was presented Wednesday 2 July 1150 by Jill Boyce:

Summary of HLS adoptions

VSEI v4:

• Mostly small editorial improvements, additions of constraints, clarifications, and bug fixes
• New feature: DSC SEIs support for signing of SEI messages 
• More significant bug fixes: GFV chroma key

HEVC:

• Editorial improvements and bug fixes

TuC:

• New features, editorial improvements, additions of constraints, clarifications, and bug fixes of existing SEI messages in TuC
• Extensions to existing SEI messages in VSEI v3
• New SEI messages:
  • Auxiliary sampling alignment and transformation information SEI 
  • Localization and mapping SEI

Adoptions List

VSEI v4

Multiple SEI messages

• JVET-AM0189 AHG9: On semantics of persistence flag of SEI messages in VSEI v4 and v3 
• JVET-AM0174 AHG9: On general SEI payload constraints 
• JVET-AM0047 AHG9: Miscellaneous changes for VSEI 
• JVET-AM0052 AHG9: Editorial changes for VSEI
• JVET-AM0191 AHG9: Editorial updates for VSEI v4 
• JVET-AM0165 AHG9: On value range and reserved values for syntax elements in VSEI v4 

SEI processing order (SPO) and processing order nesting (PON)

• JVET-AM0292 AHG9/AHG2: On the semantics of po_num_bits_in_prefix_indication_minus1[ i ] and po_sei_prefix_data_bit[ i ][ j ]
• JVET-AM0081 AHG9: On the PON dependency constraint 
• JVET-AM0082 AHG9: On the PON nested FGC SEI message
• JVET-AM0121 AHG9: On the SEI processing order SEI message 
• JVET-AM0166 AHG9: Miscellaneous aspects in VSEI v4 
• JVET-AM0321 AHG9: On SEI message types indicated by an SPO SEI message 

AI usage restrictions (AUR)

• JVET-AM0112 AHG9: On AI usage restrictions SEI message 
• JVET-AM0152 AHG9: AI usage restrictions for entities other than decoded pictures 
• JVET-AM0117 AHG9: On the AI usage restrictions SEI message 

Digitally signed content (DSC)

• JVET-AM0118 AHG9: Digital signing of selected SEI messages 
• JVET-AM0119 AHG9: On start and end flags of digitally signed 
• JVET-AM0164 AHG9: On DSC SEI 
• JVET-AM0190 AHG9: On implicit association mode in digitally-signed content SEI messages in VSEI v4 
• JVET-AM0210 AHG9: On miscellaneous aspects in digitally-signed content SEI messages in VSEI v4 draft 

Packed regions info (PRI)

• JVET-AM0075 AHG9: On packed regions information SEI message 
• JVET-AM0211 AHG9: On the packed regions information SEI message 
• Generative face video (GFV)
• JVET-AM0334 HG9: Common specification text for GFV SEI message

HEVC

• JVET-AM0189 AHG9: On semantics of persistence flag of SEI messages in VSEI v4 and v3 
• JVET-AM0048 AHG9/AHG2: Miscellaneous changes for HEVC 
• JVET-AM0324 Correction to the Alpha Channel Information SEI message processing order 

TuC

Encoder optimization info (EOI)

• JVETAM0126 AHG9: Temporal quality indication for EOI SEI message 

Scalability dimension info (SDI) SEI

• JVET-AM0090 AHG9: SDI SEI extension for signalling described layer of confidence layer 

Shutter interval

• JVET-AM0100 AHG9: Shutter interval information SEI message extension for rolling shutter cameras 

Alpha channel info (ACI)

• JVET-AM0324 Correction to the Alpha Channel Information SEI message processing order 

NNPF

• JVET-AM0087 AHG9: Comments on VSEI TuC 
• JVET-AM0173 AHG9: NNPFA SEI message extension for multi-purpose NNPFs 

Constituent rectangles (CR)

• JVET-AM0092 AHG9: Editorial changes on the CR SEI 
• JVET-AM0093 AHG9: On the CR SEI message 

Display rectangles (DR)

• JVET-AM0095 AHG9: On the DR SEI message 
• JVET-AM0146 AHG9: On display rectangles with same display aspect ratio as cropped decoded picture 

Digitally signed content (DSC)

• JVET-AM0193 AHG9: On digitally-signed content SEI messages for subpicture support in TuC of VSEI 

Film grain regions characteristics (FGRC)

• JVET-AM0085 AHG9/AHG13: On Film Grain Regions Characteristics SEI message 
• JVET-AM0124 [AHG9/AHG13]: On typo fix and interface software contribution to the FGRC SEI message 

Lens optical correction (LOC)

• JVET-AM0102 AHG9: Additional models for the LOC SEI message 
• JVET-AM0101 AHG9: On Lens Optical Correction SEI message 

Display overlays info (DOI)

• JVET-AM0094 AHG9: On the DOI SEI 
• JVET-AM0147 AHG9: On applying display overlays on display rectangles 
• JVET-AM0180 AHG9: Overlay purpose indicator for Display overlays information SEI message 

Picture segment info (PSI)

• JVET-AM0097 AHG9: On the PSI SEI message 
• JVET-AM0192 AHG9: On picture segmentation SEI message in TuC of VSEI 

Danmu information

• JVET-AM0154 AHG9: On danmu information SEI message 
• JVET-AM0161 AHG9: On display timing in the danmu information SEI message 
• JVET-AM0182 AHG9: On the Danmu SEI message 

Photosensitive content

• JVET-AM0086 AHG9: On Photosensitive Content Information Signalling 

Quality metric QM

• JVET-AM0096 AHG9: On the QM SEI message 

Bitdepth range info (BRI)

• JVET-AM0099 AHG9: On the BRI SEI message 

Colour mapping info (CMI)