JVET-AF0002 JVET AHG report: Draft text and test model algorithm description editing (AHG2) [B. Bross, C. Rosewarne (co-chairs), F. Bossen, A. Browne, S. Kim, S. Liu, J.-R. Ohm, G. J. Sullivan, A. Tourapis, Y.-K. Wang, Y. Ye (vice chairs)]

Output documents produced

JVET-AE1004 - Errata report items for VVC, VSEI, HEVC, AVC, and Video CICP

This document contains a list of reported errata items for VVC, VSEI, HEVC, AVC, Video CICP, and the TR on usage of video signal type code points, for tracking purposes. Some of the items have been confirmed by the JVET and have been agreed to require fixing. This document also provides publication status backgrounds of these standards.

Incorporated items at the JVET-AE meeting:

• Updated the general descriptions for VVC, VSEI, HEVC, and AVC.
• Removed obsolete/resolved errata items.
• Added 16 items for the ISO text only for HEVC, identified when working on the ITU-T consent text for H.265 version 9 (edition 5).
• Added 9 items on the semantics of post-filter hint SEI message from JVET-AE0155 for both HEVC and AVC.
• Added 4 items for AVC, on semantics of the omnidirectional video SEI messages. These items were among the items for HEVC and have been resolved for HEVC.
• Added an item for AVC on use of the "should" wording for the use of the film grain characteristics SEI messages when present.
• Removed the section for codepoint usage TR as the TR part has been dropped from the title of the document a while ago.
• Added an item for HEVC regarding replacing of "Max slice segments per picture MaxSliceSegmentsPerPicture" in Table A.8 with "Max # of slice segments per picture MaxSliceSegmentsPerPicture" (reported by Cliff Reader – thanks!).

JVET-AE1006 - New profiles, colour descriptors, and SEI messages for HEVC (draft 1)

This document contains the draft text for changes on new profiles, colour descriptors, and SEI messages for the 5^th edition of the High Efficiency Video Coding (HEVC) standard (Rec. ITU-T H.265 | ISO/IEC 23008-2). The changes include: 1) the support of four new profiles, namely the Multiview Main 10, Multiview Monochrome, Multiview Monochrome 10, and Multiview Monochrome 12 profiles; 2) the support of three additional colour type identifiers; 3) the HEVC-specific supports for some supplemental enhancement information (SEI) messages that may be included in HEVC bitstreams but are not to be specified in the HEVC specification, and 4) some technical corrections and editorial improvements to the 5^th edition text of HEVC. The SEI messages are the neural network post-filter characteristics (NNPFC) SEI message, the neural-network post-filter activation (NNPFA) SEI message, and the phase indication SEI message, that are to be specified in the 3^rd edition of the Versatile Supplemental Enhancement Information messages for coded video bitstreams (VSEI) standard (ITU‑T H.274 | ISO/IEC 23002-7).

Draft 1 incorporated items:

• Changes for NNPF and phase indication SEI messages (based on JVET-AE0101 as well as the latest changes to the interface text for the NNPF SEI messages agreed at the JVET-AE meeting
• JVET-AD1008 Additional colour type identifiers
• Fixes and changes for the film grain characteristics SEI message
• New multiview profiles from JVET-AE0296, plus an additional Multiview Monochrome 10 profile
• Some errata items to subclauses A.4.1, A.4.2, G.11.2.3, and H.11.2.2 reported by Cliff Reader (thanks!)
• Changes to D.1 related to referring to VSEI in HEVC

JVET-AE1016 – AVC with extensions and corrections (Draft 1)

This document contains the draft text for changes of the Advanced Video Coding (AVC) standard (Rec. ITU-T H.264 | ISO/IEC 14496-10). Text modifications are provided for 1) the support of additional colour type identifiers; 2) some technical corrections and editorial improvements to the 10^th edition text of AVC; and 3) the AVC-specific supports for some supplemental enhancement information (SEI) messages that may be included in AVC bitstreams but are not to be specified in the AVC specification. These SEI messages include three other SEI messages, namely the neural network post-filter characteristics SEI message, the neural-network post-processing filter activation SEI message, and the phase indication SEI message, that are to be specified in a new edition of the Versatile Supplemental Enhancement Information messages for coded video bitstreams (VSEI) standard (ITU‑T H.274 | ISO/IEC 23002-7).

The affected sections are provided relative to the 2022 edition of ISO/IEC 14496-10, i.e. the 10^th edition.

Draft 1 incorporated items:

• JVET-AD1008 Additional colour type identifiers for AVC
• JVET-AE0101 Use of neural network post filter and phase indication SEI messages for AVC
• JVET-AE1004 Errata items on AVC
• Fix for film grain synthesis equation in FGC SEI message (see notes under BoG report JVET-AE0294)

JVET-AE2005 - New level and systems-related supplemental enhancement information for VVC (Draft 6)

This document contains a draft amendment for changes to the Versatile Video Coding (VVC) standard (ITU‑T H.266 | ISO/IEC 23090-3). This amendment includes the following: 1) the support of an unlimited level for video profiles; 2) some technical corrections and editorial improvements to the 2^nd edition text of VVC; and 3) the VVC-specific supports for some supplemental enhancement information (SEI) messages that may be included in VVC bitstreams but are not to be specified in the VVC specification. These SEI messages include two systems-related SEI messages, one for signalling of "green metadata" as to be specified in ISO/IEC 23001-11 and the other for signalling of an alternative video decoding interface for immersive media as to be specified in ISO/IEC 23090-13, and four other SEI messages, namely the shutter interval information SEI message, the neural network post-filter characteristics SEI message SEI message, the neural-network post-processing filter activation SEI message, and the phase indication SEI message, that are to be specified in a new edition of the Versatile Supplemental Enhancement Information messages for coded video bitstreams (VSEI) standard (ITU‑T H.274 | ISO/IEC 23002-7).

Draft 6 incorporated items:

• Fixed bug tickets #1594, #1602 and #1606.
• Allow multiple NNPFC and NNPFA SEI messages with different content in the same picture unit by excluding NNPFC and NNPFA SEI messages from the existing constraint in JVET-AD2005, subclause D.3.1. The NNPFA SEI message is constrained in a manner that the payload shall be identical for the same nnpfa_target_id value. Editors are requested to make sure the blanket constraint in VVC on constraining SEI payload content to be the same within a PU does not impose a problem for SEI messages that contain ID values. No such blanket constraint appears to exist in HEVC, and this was intentional for some SEI messages such as tone mapping information and colour remapping information SEI messages. (JVET-AE0049 item 1).
• Constrain the repeated inference to generate interpolated pictures up to the end of the bitstream only to NNPFs that perform only picture rate upsampling. (JVET-AE0049 item 2)
• Change the index range in input picture assignment for repeated inference at the end of a bitstream: it is proposed to remove "− j" from "When numInputPics is greater than 1, the following applies for each value of i in the range of j + 1 to numInputPics − j − 1, inclusive, in increasing order of i:" (JVET-AE0049 item 3, JVET-AE0142 item 1)
• In the following sentence in the NNPFC interface text, replace "inputPic" with "resampledPic": If sourceWidth is equal to CroppedWidth and sourceHeight is equal to CroppedHeight, inputPic is set to be the same as sourcePic. (JVET-AE0142 item 2)
• In the NNPFC interface text, condition the following constraint on "when pictureRateUpsamplingFlag is equal to 1", and num_interpolated_pics[ i − 1 ] shoud be nnpf_interpolated_pics[ i − 1 ]: It is a requirement of bitstream conformance that num_interpolated_pics[ i − 1 ] shall not be greater than 0. (JVET-AE0142 item 3)
• VVC changes corresponding to the addition of nnpfa_no_prev_clvs_flag. (JVET-AE0050 item 1)
• VVC changes corresponding to the addition of nnpfa_no_foll_clvs_flag. (JVET-AE0050 item 2)
• At the end of a bitstream, add repeated inference of NNPF for creating pictures corresponding to input pictures (regardless of the purpose indicated by nnpfc_purpose except for picture rate upsampling) when there are multiple input pictures to which the NNPF applies, with an update to use the nnpfa_output_flag rather than the nnpfc_input_pic_output_flag. (JVET-AE0134 item 2)
• Add a constraint based on nnpfa_output_flag[ i ], as part of the process for derivation of the NNPF input pictures in the interface text, to ensure that input pictures that are not present in the bitstream do not have a corresponding NNPF output picture. (JVET-AE0070 item 2)
• Editorial changes (JVET-AE0127 attachment)
• Do not include the post-filter hint SEI message in VVC v3 due to its apparent lack of clarity. (JVET-AE0155)

JVET-AE2006 - Additional SEI messages for VSEI (Draft 5)

This document contains the draft text for changes to the versatile supplemental enhancement information messages for coded video bitstreams (VSEI) standard (Rec. ITU-T H.274 | ISO/IEC 23002-7), to specify additional SEI messages, including the shutter interval information SEI message, neural-network post-filter characteristics SEI message, neural-network post-filter activation SEI message, phase indication SEI message, and post-filter hint SEI message.

Changes that had been integrated for VSEI v3 and JVET-AE2006:

• Add missing value ranges for resampling syntax elements nnpfc_pic_width_num_minus1, nnpfc_pic_width_denom_minus1, nnpfc_pic_height_num_minus1, and nnpfc_pic_height_denom_minus1 to be in the range of 0 to 65535, inclusive. (JVET-AE0048 item 1)
• Add missing value ranges for nnpfc_inp_format_idc and nnpfc_out_format_idc to be in the range of 0 to 255, inclusive. (JVET-AE0048 item 2).
• Arrange the definitions of DeriveInputTensors(), the syntax elements related to overlap and padding type, and the definition of the InpSampleVal function so that forward references are avoided by moving the derivations of strengthControlScaledVal (Formula 87), DeriveInputTensor (Formula 88), and StoreOutputTensor (91) to a later position that immediately follows the derivation of the InpSampleVal function. (JVET-AE0048 item 3)
• Clarify that indicated or inferred values of vui_colour_primaries, vui_transfer_characteristics, vui_matrix_coeffs, and vui_full_range_flag apply when nnpfc_separate_colour_description_present_flag is equal to 0. Here, the possibility of inferring these syntax elements is made explicit when compared to JVET-AD2006. (JVET-AE0048 item 4)
• Signal nnpfc_chroma_loc_info_present_flag only when nnpfc_out_order_idc is not equal to 0. Infer nnpfc_chroma_loc_info_present_flag to be equal to 0 when not present. (JVET-AE0060 item 1)
• Clarify using the approach in the following suggestion: “nnpfc_mode_idc equal to 0 indicates that the stuff is in the SEI message. nnpfc_mode_idc equal to 1 indicates the stuff is in a location identified by a URI. When nnpfc_mode_idc is equal to 0, the stuff is in the format specified by ISO/IEC 15938-17.” (JVET-AE0175)
• Change nnpfc_reserved_zero_bit(_a/_b) to nnpfc_alignment_zero_bit(_a/_b). (JVET-AE0126 item 2)
• Change the condition for the constraint on either nnpfcOutputPicWidth is not equal to CroppedWidth or nnpfcOutputPicHeight is not equal to CroppedHeight from "when nnpfc_pic_width_num_minus1, nnpfc_pic_width_denom_minus1, nnpfc_pic_height_num_minus1, and nnpfc_pic_height_denom_minus1 are present" to "when resolutionResamplingFlag is equal to 1". (JVET-AE0126 item 5)
• Move nnpfa_target_base_flag to be immediately after nnpfa_persistence_flag, such that the persistence flag, when present, immediately follows the cancel flag. (JVET-AE0126 item 7)
• State that “All NNPFC SEI messages in a CLVS that have a particular nnpfc_id value and nnpfc_base_flag equal to 1 shall have identical SEI payload content.” (or editorial equivalent) (JVET-AE0128 item 2)
• Remove ChromaFormatIdc from the constraint for when colourizationFlag shall be equal to 0. (JVET-AE0141)
• Cancel the persistence of an NNPFA SEI message with any subsequent NNPFA SEI message with the same nnpfa_target_id. (JVET-AE0048 item 5)
• Update the constraint related to nnpfcTargetPictures and nnpfaTargetPictures by also considering the value of nnpfa_target_base_flag. (JVET-AE0189 item 3)
• Add the following constraint: When nnpfa_target_base_flag in an NNPFA SEI message is equal to 0, there shall be at least one NNPFC SEI message with nnpfc_id equal to nnpfa_target_id and nnpfc_base_flag equal to 0 that precedes the NNPFA SEI message in decoding order. (JVET-AE0189 item 2)
• Add nnpfa_no_prev_clvs_flag to the NNPFA SEI message. (JVET-AE0050 item 1)
• Add nnpfa_no_foll_clvs_flag to the NNPFA SEI message, gated by nnpfa_persistence_flag being equal to 1. (JVET-AE0050 item 2)
• Disallow generating NNPF output pictures between any particular pair of consecutive input pictures more than once. (JVET-AE0135 item 2)
• Add a constraint to ensure that there is at least one NNPF output picture when an NNPF is activated and applied. (JVET-AE0069)
• Specify that all NNPFC SEI messages with a particular value of nnpfc_id within a CLVS shall have the same value of nnpfc_purpose. (JVET-AE0142 item 4)
• Do not include the post-filter hint SEI message in VSEI v3 due to its apparent lack of clarity. (JVET-AE0155)
• Editorial changes (JVET-AE0048 item 6, JVET-AE0126 items 3, 4, 6, JVET-AE0126 attachment, and ticket #1605, JVET-AE0054)
• Use “should” for film grain synthesis and retain prior drafted language for other SEI messages. (JVET-AE0155)
• Editors are requested to make sure it is clear that activating an “update” can (and basically always would) require using information from a “base” filter in order to interpret the update. (JVET-AE0189 item 1)
• Clarify, as necessary, that an NNPFA message can activate a base with a particular ID when an update is active, which will switch the filter from the updated filter to the base filter. (JVET-AE0173)
• Add a NOTE to clarify an implied, hidden constraint on picture-rate-upsampling NNPFs with persistent activation, with expansion to consider all possible cases that have a similar implication: (JVET-AE0135 item 1)
• Editors are requested to check/clarify the existing intent that if the current picture contains an SEI message (any SEI message that uses the typical persistence scope scheme) with a cancel flag equal to 1, the current picture is not within the scope of the previous SEI message (regardless of whether the SEI message is a suffix or prefix SEI message). (JVET-AE0048 item 5)
• Text related to the remaining revisit from the BoG report JVET-AE0272 was presented on Tuesday 18 July at 1035. This is related to the issue that for frame interpolation between frames A and B only one activation shall be performed for all frames that are to be interpolated altogether. It was commented that the purpose may be difficult to understand for implementers. It was suggested to add a NOTE explaining the purpose of that constraint.

JVET-AE2027 - SEI processing order SEI message in VVC (draft 5)

This document contains the draft text for changes to the Versatile Video Coding (VVC) standard (Rec. ITU-T H.266 | ISO/IEC 23090-3), to specify the SEI processing order SEI message.

Changes yet to be integrated:

None.

Changes that had been integrated:

Updates from JVET-AE0156. Leave out the change in CVS (for further study), and add a note about the usage of wrapping.

Related input contributions

The following input contribution was noted as relevant to the work of this ad hoc group:

• JVET-AF0045 AHG2/AHG9: On the use of NNPF SEI messages in AVC
• JVET-AF0063 On MV-HEVC profiles
• JVET-AF0064 AHG2: Some editorial issues in AVC, HEVC, VVC, and VSEI

Remaining bug tickets

• #1594 Mismatch between VVC spec and VTM for sample generation in CCLM process
• #1607 Wrong sign (+ instead of -) in Fig 16 Flowchart for decoding a decision in the ITU text
• #1609 NoBackwardPredFlag derivation ambiguity
• #1617 Not initialized NumCtusInSlice[0] to 0.

Recommendations

The AHG recommended to:

• Approve JVET-AE1004, JVET-AE1006, JVET-AE1016, JVET-AE2005, JVET-AE2006, and JVET-AE2027 documents as JVET outputs,
• Compare the VVC documents with the VVC software and resolve any discrepancies that may exist, in collaboration with the software AHG,
• Encourage the use of the issue tracker to report issues with the text of both the VVC specification text and the algorithm and encoder description,
• Continue to improve the editorial consistency of VVC text specification and Test Model documents,
• Ensure that, when considering changes to VVC, properly drafted text for addition to the VVC Test Model and/or the VVC specification text is made available in a timely manner,
• Review bug tickets, and other AHG2 related inputs and act on them if found to be necessary.