JVET-T0048 On the syntax design for extending SEI messages and VUI for HEVC, VVC and VSEI [Y.-K. Wang (Bytedance), G. J. Sullivan (Microsoft)]

This contribution was discussed at 1900 Wednesday 7 October 2020 (chaired by JRO).

This contribution discusses the SEI payload syntax in AVC, HEVC, and VVC/VSEI, and the VUI payload syntax in VVC/VSEI. It reports an asserted bug on the HEVC buffering period SEI message syntax extension and proposes a fix for the asserted bug, and discusses how extensions, when needed, should be specified for the syntax of SEI messages in HEVC and VSEI, and to the syntax of VUI in VSEI.

The first method was suggested to be “cleaner”, future proof, and easier to understand, while the second would isolate the changes to the one place where there is an asserted problem in HEVC.

For VVC, it was asserted that the first proposed approach would enable detection in the VVC spec of whether the VSEI spec had been extended.

It was discussed whether a third approach could be used, or a NOTE could be sufficient to clarify the corner case. It was sugested that the problem could exist only if the use_alt_cpb_params_flag is equal to 1.

The existing extension mechanism for SEI messages (which is there from HEVC and was also used in VVC/VSEI) was reported to cause a problem, as an extension of less than one byte with a pattern 10..0 would be interpreted as a byte alignment mechanism. The issue could only occur for very short extension payloads, as otherwise the presence of the extension would be identified by the payload size. Currently, only the buffering period SEI message could have a case where it is affected.

Two possible solutions are suggested: a) Define a longer payload extension for the SEI messages that are potentially affected (“second method”). b) Define for the buffering period SEI message (and for future SEI messages that might have a similar problem) an extra flag variable that clearly signals the presence of an extension (“first method”).

During the discussion, it is suggested that another solution could be to put a note into the spec. pointing out this case for buffering period. This would not affect existing implementations, but make encoder manufacturers aware that a decoder would ignore an intended extension in this corner case (which a more smart encoder could avoid). This would however not avoid similar cases in newly defined SEI messages (if not carefully designed, or proponents of such SEI not being aware of this problem), for future extensions the “first method” appears to be cleaner and better to understand.

Offline study was requested and this was further discussed in session 23 at 0730 on 16 October (chaired by JRO).

A third possible expression is to add the check for more_data_in_payload() before payload_extension_present() and state in HEVC that it is a requirement of bitstream conformance that when use_alt_cpb_params_flag is present, that more_data_in_payload() shall be equal to 1 in the SEI payload syntax structure. The proponents said that this approach is technically equivalent to the other two proposed approaches, but has the minimum visible impact on the specification.

Decision (BF): Adopt the third approach for HEVC and proactively adopt the “first method” (approach b) into the working drafts toward v2 of VVC and VSEI.