JVET-AQ0086 AHG2/AHG9: Initial proposed text for a potential WD of VSEI v5 [Y.-K. Wang, J. Xu, L. Zhang, K. Zhang (Bytedance), S. Deshpande, J. Samuelsson-Allendes (Sharp), Y. He, L. Kerofsky (Qualcomm), A. M. Tourapis, D. Podborski, J. Kim (Apple), E. François, C.-H. Demarty, A. Ak (InterDigital), S. Wenger, G. Teniou (Tencent), H. Tan (LGE)]
This contribution proposes, if the JVET decides to start a WD of VSEI v5 at this meeting, an initial proposed text for the potential WD of VSEI v5. The proposed initial text includes the following two categories of items from the VSEI TuC in JVET-AP2032:
– Category 1 (clear yes for inclusion into the WD; the proponents think that for these items, the use cases look clear, the designs look good and the texts look mature, and software and meaningful showcases have been provided):
– Extensions to the scalability dimension information (SDI) SEI message
– Extensions to the shutter interval information (SII) SEI message
– The film grain regions characteristics (FGRC) SEI message
– The constituent rectangles (CR) SEI message
– The localization and mapping (LAM) SEI message
– The photosensitive content information (PCI) SEI message
– The picture segmentation information (PSI) SEI message
– The danmu information (DI) SEI message
– The colour mapping information (CMI) SEI message
– The enhanced colour format information (ECFI) SEI message
– Category 2 (yes for inclusion into the WD if implementations and meaningful showcases are provided at this meeting; the proponents think that for these items, the use cases look clear, the designs look good and the texts look mature, but software and meaningful showcases are yet to be provided):
– Extensions to the encoder optimization information (EOI) SEI message
– Extensions to the digitally signed content (DSC) SEI messages
– Extensions to the packed regions information (PRI) SEI message
The following three attachment files are included in this contribution:
– The first file contains text changes for all category 1 items, with change marks relative to the latest ITU-T published text of H.274/VSEI v4.
– The second file shows change marks for the new SEI messages relative to just copying and pasting the syntax and semantics from the TuC. All these changes are asserted to be purely editorial improvements. Since the style of each syntax row in the syntax tables has been changed to be consistently using the style named "table syntax", there are lots of formatting changes shown. In Word, under "Review", you can click "Show Markup" and check off "formatting", then all formatting change marks won’t be shown.
– The third file contains text changes for the three category 2 items on top of changes for all category 1 items in the first file, using a different user ID such that these changes for the category 2 items can be easily identified.
It is suggested to start with a VSEI WD in the form of fully-integrated text instead of the amendment form. An advantage of this is that while during the development of the new version, hopefully a good full text is gradually developed, thus avoiding the last-minute hassle of integrating all amendment deltas, in which case some of the items can easily get lost. The provided texts are suitable for this purpose, as the used basis texts were the latest ITU-T published texts.
However, if the group decides to go with the amendment form, the proponents are happy to generate whatever agreed parts in the amendment form.
Initial discussion at plenary level Tuesday 7 July 1910-2000 (chaired by JRO).
The TuC JVET-AP2032 contains 23 new messages (including 3 on GSC), and extensions to 8 existing messages (where NNPF has 6 different extensions, and SPO has 2 different extensions).
As an initial step, all 12 contributions in this category were requested to provide a short description of the use case for each item (new message or extension) for further discussion.
Use cases might also be added to the table of the document 2032 (or a corresponding table to be created in the WD), unless the usage/showcase in the rightmost column is sufficient.
It is noted that the table may require some update on software availability.
It was commented that starting such a work item as new edition may not be possible in ISO, as edition 4 FDIS is still under ballot, so publication may not be before 2027.
Follow-up discussion Thursday 9 July 1115:
1) Extensions to the scalability dimension information (SDI) SEI message
As per criteria:
- Maturity: good
- Usage scenario: clear, includes also new aux pic types to signal confidence of another aux pic, which is useful e.g. for depth, and segmentation by signalling labels. Semantics of confidence may require some more precise definition in terms what confidence is expressing, which metric (probability, error in measurement), etc. is expressed by the confidence
- Showcase: n/a
- Software: not available for confidence, available for segmentation, write/parse of SDI would need to be extended by new types
- Added value in SEI universe: clear
- Added value compared to systems level solutions: n/a
- Competition with other developments: n/a
The film grain regions characteristics (FGRC) SEI message
As per criteria:
- Maturity: good
- Usage scenario: clear
- Showcase: available JVET-AH0112
- Software: available for write/parse and synthesis of film grain, extraction of parameters would be desirable
- Added value in SEI universe: clear
- Added value compared to systems level solutions: n/a
- Competition with other developments: n/a
The constituent rectangles (CR) SEI message
As per criteria:
- Maturity: good
- Usage scenario: clear
- Showcase: available by demonstrating picture examples in JVET-AH0162
- Software: available for write/parse and extraction of rectangles from decoded output pictures
- Added value in SEI universe: more flexible usage than using subpictures
- Added value compared to systems level solutions: n/a
- Competition with other developments: n/a
The picture segmentation information (PSI) SEI message
As per criteria:
- Maturity: good
- Usage scenario: clear
- Showcase: available in JVET-AP0245 (part on picture segmentation tool)
- Software: available for write/parse and extraction or marking of segments from decoded output pictures; encoder input to aux picture needs to be prepared separately
- Added value in SEI universe: benefit over OMI should be clarified
- Added value compared to systems level solutions: n/a
- Competition with other developments: n/a
The danmu information (DI) SEI message
As per criteria:
- Maturity: good
- Usage scenario: clear
- Showcase: available in JVET-AP0097
- Software: available for write/parse, sufficient for the purpose
- Added value in SEI universe: clear
- Added value compared to systems level solutions: more flexible than subtitles used somewhere which is not supporting larger number or dynamically moving text in video in an efficient way
- Competition with other developments: n/a
The colour mapping information (CMI) SEI message
As per criteria:
- Maturity: good
- Usage scenario: clear
- Showcase: available in JVET-AO0166
- Software: available for write/parse, and processing of decoder output
- Added value in SEI universe: mapping of monochrome to three-component pictures not available in any other SEI message
- Added value compared to systems level solutions: n/a
- Competition with other developments: n/a
The enhanced colour format information (ECFI) SEI message
As per criteria:
- Maturity: good
- Usage scenario: clear
- Showcase: available by demonstrating picture examples in JVET-AI0257
- Software: available for write/parse, generation of input pictures, extraction of packing parameters, and processing of decoder output
- Added value in SEI universe: clear
- Added value compared to systems level solutions: n/a
- Competition with other developments: n/a
Extensions to the digitally signed content (DSC) SEI messages
As per criteria:
- Maturity: not fully clear yet, various new input contributions on other aspects
- Usage scenario: clear
- Showcase: available in JVET-AQ0115
- Software: available for write/parse and marking parts of picture from showcase
- Added value in SEI universe: clear
- Added value compared to systems level solutions: n/a
- Competition with other developments: n/a
Extensions to the packed regions information (PRI) SEI message
As per criteria:
- Maturity: clear
- Usage scenario: clear
- Showcase: not available; would be good to demonstrate that mixed content can be improved by coding the screen content part in 4:4:4 in a mixed content image at same bit rate
- Software: not available, would be needed for showcase
- Added value in SEI universe: may depend on showcase
- Added value compared to systems level solutions: n/a
- Competition with other developments: n/a