JVET-AO0212 Extension mechanism for new AVC SEI messages [J. Samuelsson-Allendes, S. Deshpande (Sharp), Y.-K. Wang (Bytedance)]

In this input document, an extension mechanism is proposed to be added to the SEI payload syntax structure in AVC. The proposed extension mechanism is identical to the extension mechanism that exists in VVC. The extension mechanism is proposed to be present only for SEI messages added in the next version (and future versions) of the AVC standard. Existing SEI messages and legacy AVC decoders are not impacted by the proposed extension mechanism.

This contribution is proposed for the upcoming new version of AVC, which will reference VSEI v4. It would apply to SEI message in VSEI v3.

The NNPFA/NNPFC messages are in a published version of AVC, so it would not enable support for the extensions in the new version of AVC.

It was suggested to check that the proposed syntax structures are defined in AVC. If not, they would need to be added.

There may be other changes needed in VSEI.

It was suggested that an additional payload type could be assigned in AVC for extensible/extended versions vs. non-extensible/un-extended versions of the same SEI message.

The current proposal has a condition payloadType <= 212. It is noted that specific payloadType values could be listed without restricting them to a range.

It was noted that we have attempted to match payload type values for the same SEI message across standards.

It was suggested that the proposed syntax table could be restructured to have common syntax elements for bit_equal_to_one and bit_equal_to_zero.

There was some support initially expressed for the proposal, further discussion was requested.

A -v2 version adds text from JVET-AO0280.

It is suggested to be modified to include a list of payload types instead of a less than condition. It is also suggested that the additional payload types allocated to the FGC and NNPFA SEI message with values in the AVC range.

This was further discussed at 1400 on Thursday 22 Jan 2026.

A -v3 version was uploaded, and a -v4 version is to be uploaded.

It was agreed to add this to the AVC output documents in both JVET-AO1016 and JVET-AO1017.

JVET-AO0212 Extension mechanism for new AVC SEI messages

5TuC

See list under JVET-AO2032

No new messages

In the context of discussion mandates of AHG7, it was suggested that it might be desirable to conduct investigation with non-CTC sequences, potentially under CfE/CfP conditions (rate matching not necessary).

MPEG information sharing meetings

Information sharing sessions with other WGs and AGs of the MPEG community were held on Monday 19 Jan. 0500–0800, Wednesday 21 Jan. 0500–0600, and Friday 23 Jan. 2100–2300.

The status and plans for the work in the MPEG WGs and AGs was reviewed at these information sharing sessions.

Joint meetings

Joint session 1530-1620 Monday 19 Jan. on lenslet video coding: MPEG WG 4 / Video, MPEG WG 5 / JVET, and VCEG (ITU-T Q6/21)

The session description is based on notes by G. J. Sullivan.

This joint session was chaired by Lu Yu (MPEG WG 4 Convenor), Jens-Rainer Ohm (JVET Chair), Gary J. Sullivan (SC 29 Chair & Q6/21 Rapporteur), Jörn Ostermann (MPEG AG 2 Convenor), and Yan Ye (Q6/21 Associate Rapporteur).

WG 4 wants to create a new standard for lenslet video coding (LVC).

Additional coding tools are considered possible to be specified for LVC, but are planned to be outside of the core coding loop.

Thus, the new standard would be a post-processor (from the perspective of the “inner decoder”), producing transformed output pictures from each inner decoder’s output picture.

Some HLS would be added/inserted in the bitstream.

There was discussion about whether it is necessary to touch parameter sets, or if this be supported by SEI or new NAL unit types.

Compatibility with existing decoders was suggested to be a concern.

If defined as a new profile of HEVC or VVC (in the ordinary way), that could cause existing decoders to reject the bitstream.

A bitstream extension mechanism is proposed in JVET-AO0284. See the summary of that document provided elsewhere in this report.

As contrasted with the approach taken for MIV, V-PCC, and VCM, in which an HEVC or VVC bitstream would be encapsulated into some other bitstream that would not conform to HEVC or VVC, it is proposed to put extension data within an HEVC or VVC bitstream. These alternative approaches are illustrated in the figure below, where the left side depicts the approach taken for V-PCC, MIV, and VCM, and the right side shows the approach proposed in JVET-AO0284 with embedding of extension data within an HEVC or VVC bitstream.

The proposal would identify a short bit pattern as an extension type code for parameter sets.

It was commented that a similar extension mechanism had previously been proposed.

There was discussion about whether the base video standard should identify what the particular pattern means.

It was asked whether the parameter sets really need to be changed. Several participants said that SEI might be a better place to specify a post processing syntax.

It was also suggested that we could just use one reserved bit as the extension mechanism if a parameter set extension is needed.

It was asked what would be the expected behaviour of an existing VVC or HEVC decoder that encounters the extended bitstream. It was commented that the scheme would be similar to frame packing – with no impact on low-level decoding process.

This would likely not be a new profile from the HEVC or VVC perspective.

Using this scheme could involve defining an extension of every relevant video coding standard (e.g., AVC, HEVC, and VVC).

It was asked how much extension data is involved.

It was commented that parameter sets are ordinarily for syntax relating to the inner decoding process, not post-processing.

It was suggested that post-processing could be normative even if this is done with an SEI message (e.g., like Green metadata, which would not necessarily need to be directly specified in the VSEI standard).

WG 4 suggested not to mention this extension in the relevant core video coding standards. Others thought that at least identifying the extension code and payload size would be needed, saying there is potential for so-called “collisions” in the future to occur.

The proposed extension patterns are currently specified in the core video coding standards as reserved and not to be used.

It was suggested that the interaction with layered coding and with schemes such as rectangle extractions and compositing would need to be considered.

It was suggested that integrating a specified functionality, possibly with a large amount of extra data, should be considered, together with schemes such as film grain synthesis.

Further study seemed needed. An SEI approach such as with Green metadata should be further studied.

SEI with normatively specified post-processing was suggested to be appropriate for this and other similar schemes.

Joint session 1620-1740 Monday 19 Jan. on Gaussian splat coding: MPEG WG 4 / Video, MPEG WG 5 / JVET, MPEG WG 7 3D Graphics, and VCEG (ITU-T Q6/21)

The session description is based on notes by G. J. Sullivan.

This session was chaired by Lu Yu (MPEG WG 4 Convenor), Gary Sullivan (SC 29 Chair and Q6/21 Rapporteur), Jörn Ostermann (MPEG AG 2 Convenor), Yan Ye (Q6/21 Associate Rapporteur), and Marius Preda (MPEG WG 7 Convenor).

The initial discussion included:

The incoming liaison statement from SG21.

A summary of status in MPEG WGs 4 & 7.

~7 technical documents for SEI that had been submitted.

JVET-AO0152 was suggested for focus.

Liaison statement content from ITU-T SG 21:

“This liaison statement expresses to ISO/IEC JTC 1/SC 29 and its working groups an interest in collaboration on 3D Gaussian Splat coding technologies that take advantage of software and hardware implementations of existing and future video coding standards jointly developed by ITU-T and ISO/IEC

ITU-T Study Group 21 (SG21) wishes to inform ISO/IEC JTC 1/SC 29 and its working groups of its recent discussions and developments regarding 3D Gaussian Splat coding technologies.

Significant industry interest has been voiced during our meetings in support of standardizing technologies for 3D Gaussian Splat coding, particularly taking advantage of software and hardware implementations of existing and future video coding standards jointly developed by ITU-T and ISO/IEC.

According to proponents, such industry interest can be effectively addressed using standardized supplemental enhancement information (SEI) messages, and several related contributions have been submitted to recent meetings of the ITU-T/ISO/IEC Joint Video Experts Team (JVET). This approach has been discussed within SG21, and no objections to this approach have been raised.

We suggest that an appropriate standard for such SEI messages is the Video Supplemental Enhancement Information (VSEI) standard (Rec. ITU-T H.274 | ISO/IEC 23002-7), which provides a unified framework for SEI technologies applicable to the modern video coding standards jointly developed by ITU-T and ISO/IEC. We further note that VSEI is developed and maintained within JVET.

In light of the shared interests and potential for alignment, SG21 proposes a joint discussion between SG21 and SC 29 experts during the upcoming scheduled teleconference meetings of Q6/21 and MPEG in January 2026, to explore possible coordination and exchange of technical perspectives on this topic.

We appreciate the continued collaboration between our groups and look forward to further engagement.”

This liaison statement suggests VSEI as the approach.

It was commented that SG21 might not be aware of the status in SC 29, but it was commented that the status in SC 29 was generally known to the SG21 participants.

The status in WG 4 and 7 was reviewed, with three projects and active explorations that were noted:

An amendment of V-PCC, with WG 7 planning to issue a CDAM at the current meeeting

An amendment of G-PCC, with WG 7 planning to issue a CDAM at the next meeting (in April 2026)

Additional exploration activity, targeting development of a CfP, to include dynamic splatting as well as static content representation

Guide to abbreviations:

PCC = point cloud compression

V = video-based (e.g. using VVC or HEVC)

G = geometry-based.

Gaussian splats can be interpreted as using attributes associated with points in a point cloud, where the centre of each 3D Gaussian is the corresponding point in the point cloud.

V3C is the framework used for V-PCC, MIV (MPEG immersive video) and V-DMC (dynamic mesh coding).

G-PCC has a recently added profile with extended transform precision, modified quantization, and other enhancements.

V-PCC uses four video frames (or single frames with regions) that carry the following information:

Geometry

Texture

Occupancy

Atlas data syntax, describing patches, is specified.

The principal focus of the V-PCC planned near-term amendment is static Gaussian splats, coded using video I frames / 2D still pictures.

There is also the study of dynamic (i.e., moving) Gaussian splats, planned for a future CfP.

Bit depth support was discussed; even 8-bit video approaches are considered.

JVET-AO0152 was discussed, summarized as:

This contribution argues that it is within JVET’s mandate to work on SEI messages for VSEI to interpret samples reconstructed from coded video pictures according to H.26[4/5/6] / AVC, HEVC, VVC and future video codec designs as Gaussian Splat parameters. Technical, business, and venue-related considerations are included. For this meeting, it is proposed to allow work, including proposal presentation and TuC-level decision making, in the JVET HLS breakout.

Use of a video approach was suggested by the proponent to leverage high-volume implementations of video standards.

Using VSEI was suggested by the proponent, due to its applicability to multiple video coding standards.

Basic architectural approaches as described for the lenslet handling could apply in principle, and there could be different groups where the work could be done.

It was commented that having good ways to compare approaches (e.g. common test conditions) and avoiding duplications are desirable. It was pointed out that multiple approaches are already being pursued.

It was commented that MPEG WG 2 (MPEG Requirements) is also relevant and studying related matters.

Discarding of SEI messages has been practiced. It was commented that the “no frame packing” flag can help.

Conformance specification is also a matter of concern. A conformance specification for how to use the data may be needed.

We have already agreed that, in principle, there can be normative conformance requirements that involve an SEI message (or similar kinds of data, such as reserved ignored data or other NAL units).

The mapping between the 3D representation and the 2D representation used for video/image coding was suggested to be critical. It was suggested that different encapsulation types could be specified in a harmonized way. A basic scheme could be the same aside from how the data is encapsulated.

Having joint investigation of the topic by MPEG WG 7 with JVET (as well as WG 4) was suggested, starting at the current meeting, including consideration of the 6 other noted contributions (which are not all technically distinct). Daily joint sessions of WGs 4 & 7, of 2 hours per day [1300 UTC Tuesday] had been planned already.

The extension/encapsulation method for how to carry the syntax elements that are not put into 2D picture data was suggested to be lower immediate priority than the effectiveness of the basic scheme.

Dynamic splats were suggested to be very important.

Another participant emphasized a desire to develop something relatively quickly that can be implemented with relatively low resource requirements.

It was agreed to proceed with the joint meeting session as noted and as described in the next section.

Joint session 1300-1510 Tuesday 20 Jan. on review of Gaussian splatting technical inputs: MPEG WG 4 / Video, MPEG WG 5 / JVET, and MPEG WG 7 / 3D Graphics Coding

This joint meeting with WG 4 and WG 7 was held during 1300–1510 UTC on Tuesday 20 Jan. 2026 (chaired by JRO, M. Preda and L. Yu). Notes captured during this meeting can be found in section 6.3.1.

As a follow-up action, a Joint AHG was established on the topic (see section 9)

Joint session 1430-1600 Thursday 22 Jan. on next generation video standardization Call for Proposals: MPEG WG 2 / Requirements, MPEG WG 5 / JVET, MPEG AG 5 Visual Quality Assessment, and VCEG (ITU-T Q6/21)

This joint session was chaired by Jens-Rainer Ohm (JVET Chair and WG 5 Convenor), Gary Sullivan (VCEG Rapporteur and SC 29 Chair), Yan Ye (VCEG Associate Rapporteur), Mathias Wien (AG 5 Convenor), and Jörn Ostermann (AG 2 Convenor) and Mary-Luc Champel, on behalf of Igor Curcio (WG 2 Convenor).

This section is based on notes taken by GJS.

About 300 people attended this session. The following issues were discussed:

The basic goal of the joint session was to assess whether the CfP draft was mature enough to declare and issue it as an official “Draft CfP” (not planned as final), including whether it was.

Sufficiently detailed on what will be done and when

Sufficiently aligned with requirements

The draft was presented by M. Wien and was somewhat refined in the discussion. This included review of

timelines

categories of test content

anchors

Aspects to be disclosed to proponents in advance and additional testing to be conducted later.

Possible use of expert viewing for subjective evaluation was suggested to be clarified.

Ultra-low latency and error/loss resilience and concealment – additional functionalities can be demonstrated in response to the CfP and will be considered in the collaborative phase.

Further refinement of the requirements document can be conducted later, and can hypothetically be ongoing activities.

Estimation of the expected amount of the testing fee.

Potential use of a Docker container. It was commented that this might be unnecessary. (The current Ubuntu guidance is already only a “should”.)

Requesting encoder executables and what we might do with them.

Planned subjective evaluation methodology.

The potential requirement for complete submissions for all test sequences in each test case, and what might occur if a submitter wishes to submit partial results.

Further refinements can be made until the final CfP is produced.

It was noted that further study will be conducted at an AHG meeting in February in Aachen.

In answer to the primary question of whether the CfP draft is mature enough to declare and issue it as a “Draft CfP” (not planned as final), the answer was Yes; this was approved.

An editing period for the draft was agreed – until January 30.

BoGs (2)

JVET-AO0212 Extension mechanism for new AVC SEI messages

It is noted that the list above may not be complete, and some aspects relating to VSEI interface may first need a correction in VSEIv4 (see JVET-AO1004); if some adoption is missing that is recorded somewhere else in the meeting notes it shall also be considered included.

Remains valid – not updated: JVET-AN1018 HEVC with extensions and corrections (Draft 3) [Y.-K. Wang, B. Bross, S. Deshpande, G. J. Sullivan, A. Tourapis] (2025-10-17)

Primary editor: Y.-K. Wang.

This was submitted to ITU consent on H.265 11th ed. It is noted that this may not be identical with the final version published by ITU-T.

Remains valid – not updated: JVET-AN1019 VSEI with extensions and corrections (Draft 1) [J. Boyce, J. Chen, S. Deshpande, M. M. Hannuksela, S. McCarthy, G. J. Sullivan, H. Tan, Y.-K. Wang] (2025-10-17)

Primary editor: J. Boyce.

This was submitted to ITU consent on H.274 4th ed. It is noted that this may not be identical with the final version published by ITU-T.

No output: JVET-Axx1020 through JVET-Axx1099

Remains valid – not updated JVET-AA1100 Common Test Conditions for HM Video Coding Experiments [K. Sühring, K. Sharman]

This specifies only the CTC for non-4:2:0 colour formats. The corresponding document for VVC is JVET-T2013, with no unification yet. See note under JCTVC-P1006 above.

Links to test sequences need to be updated due to the change of the content server.

Remains valid – not updated: JVET-AN2001 VVC with extensions and corrections (Draft 1) [Y.-K. Wang, B. Bross, M. M. Hannuksela, G. J. Sullivan] (2025-10-17)

Primary editor: Y.-K. Wang.

This was submitted to ITU consent on H.266 4th ed. It is noted that this may not be identical with the final version published by ITU-T