JVET-AD0009 JVET AHG report: SEI message studies (AHG9) [S. McCarthy, Y.-K. Wang (co-chairs), T. Chujoh, S. Deshpande, C. Fogg, Hendry, P. de Lagrange, G. J. Sullivan, A. Tourapis, S. Wenger (vice-chairs)]

Related contributions

A total of 41 contributions were identified relating to the mandates of AHG9. Some contributions also relate to the work of other AHGs, specifically AHG8, AHG11, and AHG13.

The number of contributions relating to each AHG9 mandate was as follows:

• 36 contributions relate to the mandate to study the SEI messages in VSEI, VVC, HEVC, and AVC. Some contributions relate to more than one SEI message.
  • 32 contributions relate to the neural-network post-filter SEI messages, including 1 summary of proposals on NNPF and SEI processing order SEI messages.
  • 5 contributions relate to the SEI processing order SEI message, including 1 summary of proposals on NNPF and SEI processing order SEI messages.
  • 1 contribution relates to the post filter hint SEI message.
  • 1 contribution relates to the annotated region SEI message
  • 1 contribution relates to the film grain characteristics SEI message
• 0 contributions relate to the mandate to collect software and showcase information for SEI messages;
• 6 contributions relate to the mandate to identify potential needs for additional SEI messages.

The following is a list of contributions related to the mandates of AHG9.

Study of the SEI messages in VSEI, VVC, HEVC and AVC

• JVET-AD0258 AHG9: A summary of proposals on NNPF and SEI processing order SEI messages [Y.-K. Wang (Bytedance)]

Neural-network post filter characteristics and activation SEI messages

Proposals on NNPF SEI messages

• JVET-AD0052 AHG9: NNPFC SEI extension mechanism to enable usages beyond user viewing [M. M. Hannuksela, F. Cricri, H. Zhang (Nokia)]
• JVET-AD0053 AHG9: On NNPFC SEI for filtering input pictures and upsampling picture rate [M. M. Hannuksela, F. Cricri, M. Santamaria (Nokia)]
• JVET-AD0054 AHG9: On NNPFC SEI for picture rate upsampling [M. M. Hannuksela, F. Cricri, M. Santamaria (Nokia)]
• JVET-AD0055 AHG9: On buffering requirements for multi-picture NNPFC SEI [M. M. Hannuksela, F. Cricri (Nokia)]
• JVET-AD0056 AHG9: Miscellaneous VSEI changes on neural-network post-filter SEI messages [M. M. Hannuksela, F. Cricri, M. Santamaria (Nokia)]
• JVET-AD0057 AHG9: Miscellaneous VVC changes on SEI messages [M. M. Hannuksela, F. Cricri, M. Santamaria (Nokia)]
• JVET-AD0058 AHG9: Approaches for region-based neural-network post-filtering [M. M. Hannuksela, F. Cricri (Nokia)]
• JVET-AD0059AHG9: Indicating processing order of NNPFs and SEI messages [M. M. Hannuksela, F. Cricri (Nokia)]

(JVET-AD0059 also relates to the SEI processing order SEI message)

• JVET-AD0065 AHG9: On NNPFC Auxiliary Input Information [S. Deshpande (Sharp)]
• JVET-AD0066 AHG9: On NNPFC Description and Identification [S. Deshpande (Sharp)]
• JVET-AD0067 AHG9: Comments on Neural-Network Post-Filter Characteristics Message [S. Deshpande (Sharp)]
• JVET-AD0078 AHG9: On comments for the draft of VSEI [T. Chujoh, Y. Yasugi, T. Ikai (Sharp)]

(JVET-AD0078 also relates to the post filter hint SEI message)

• JVET-AD0079 AHG9: On the SEI processing order SEI message [P. Yin, S. McCarthy, W. Husak, K. Konstantinides, T. Lu, F. Pu, A. Arora, T. Shao (Dolby)]

(JVET-AD0078 also relates to the SEI processing order SEI message)

• JVET-AD0087 AHG9: On cascading of post-processing filters [Y.-K. Wang, L. Zhang, J. Xu, C. Lin, Y. Li, J. Li, K. Zhang (Bytedance) Preparation of FDAM/FDIS text of VVC 2nd Ed.]

(JVET-AD0087 also relates to the SEI processing order SEI message)

• JVET-AD0088AHG9: Overall post-processing filtering process using NNPFs [Y.-K. Wang, J. Xu, L. Zhang (Bytedance)]
• JVET-AD0089 AHG9: NNPF activation parameters [Y.-K. Wang, Y. Li, J. Xu, C. Lin, J. Li, L. Zhang, K. Zhang (Bytedance)]
• JVET-AD0090 AHG9: On input pictures to an NNPF [Y.-K. Wang, L. Zhang, C. Lin (Bytedance)]
• JVET-AD0091 AHG9: Miscellaneous NNPF topics (set 1) [Y.-K. Wang, J. Xu, C. Lin, Y. Li, J. Li, L. Zhang, K. Zhang (Bytedance)]
• JVET-AD0093 AHG9: Neural-network post-processing filter with downsampling capabilities [Y. Li, J. Xu, Y.-K. Wang, C. Lin, J. Li, K. Zhang, L. Zhang (Bytedance)]
• JVET-AD0094 AHG9: On the NNPFC SEI message involving depth pictures [C. Lin, Y.-K. Wang, J. Xu, Y. Li, J. Li, K. Zhang, L. Zhang (Bytedance)]
• JVET-AD0104 AHG9: On NNPF Overlap [A. Sidiya, S. Deshpande (Sharp)]
• JVET-AD0141 AHG9: On discardable and non-output pictures for NNPF SEI messages [Hendry, J. Nam, H. Jang, S. Kim, J. Lim (LGE)]
• JVET-AD0143 AHG9: On picture size for input and output pictures of NNPFC SEI message [Hendry, J. Nam, H. Jang, S. Kim, J. Lim (LGE)]
• JVET-AD0145 AHG9: On non-consecutive input pictures for NNPFC SEI message [Hendry, J. Nam, H. Jang, S. Kim, J. Lim (LGE)]
• JVET-AD0149 AHG9: On carriage of NNPF bitstream only in NNPFC SEI message [Hendry, J. Nam, H. Jang, S. Kim, J. Lim (LGE)
• JVET-AD0151 AHG9: On output picture design in NNPFC SEI message [Hendry, J. Nam, H. Jang, S. Kim, J. Lim (LGE)]
• JVET-AD0154 AHG9: Additional purposes for the NNPFC SEI message [M. Pettersson, M. Damghanian, R. Sjöberg (Ericsson)]
• JVET-AD0159 AHG9: Explicit identification of NNPF input pictures [Rickard Sjöberg, Martin Pettersson, Mitra Damghanian (Ericsson)]
• JVET-AD0163 AHG11: Updating NNPFC and NNPFA SEI messages for the NNPF [M. Santamaria, M. M. Hannuksela, F. Cricri, R. Yang (Nokia)]
• JVET-AD0233 AHG9: Miscellaneous NNPF topics (set 2) [J. Xu, Y.-K. Wang, C. Lin, Y. Li, J. Li, L. Zhang, K. Zhang (Bytedance)]
• JVET-AD0240 AHG9: Support more NNPF purposes [J. Xu, Y.-K. Wang, L. Zhang, J. Li, C. Lin (Bytedance)]

SEI processing order SEI message

• JVET-AD0059AHG9: Indicating processing order of NNPFs and SEI messages [M. M. Hannuksela, F. Cricri (Nokia)]

(JVET-AD0059 also relates to the NNPF SEI messages)

• JVET-AD0061 AHG9: On the SEI processing order SEI message [Y. He, M. Coban, M. Karczewicz (Qualcomm)]
• JVET-AD0079 AHG9: On the SEI processing order SEI message [P. Yin, S. McCarthy, W. Husak, K. Konstantinides, T. Lu, F. Pu, A. Arora, T. Shao (Dolby)]

(JVET-AD0079 also relates to the NNPF SEI messages)

• JVET-AD0087 AHG9: On cascading of post-processing filters [Y.-K. Wang, L. Zhang, J. Xu, C. Lin, Y. Li, J. Li, K. Zhang (Bytedance)

(JVET-AD0087 also relates to the NNPF SEI messages)

Post filter hint SEI message

• JVET-AD0078 AHG9: On comments for the draft of VSEI [T. Chujoh, Y. Yasugi, T. Ikai (Sharp)]

(JVET-AD0078 also relates to the post filter hint SEI message)

Annotated regions SEI message

• JVET-AD0042 [AHG8] Proposed text for optimization of encoders and receiving systems for machine analysis of coded video content [J. Chen (Alibaba), C. Hollmann (Ericsson), S. Liu (Tencent)]

(Clause 8 discusses the annotated region SEI message)

Film grain characteristics SEI message

• JVET-AD0241 AHG13: On Film Grain Objective Metrics [X. Meng, W. Zhang, S. Labrozzi (Disney Streaming)]

Identify potential needs for additional SEI messages

• JVET-AD0051 AHG9: Common SEI Message of Generative Face Video [B. Chen, J. Chen, Y. Ye (Alibaba), S. Wang (CityU)]
• JVET-AD0060 AHG8/AHG9: Indications related to suboptimal user viewing [M. M. Hannuksela, F. Cricri, H. Zhang (Nokia)]
• JVET-AD0121 AHG9: Attenuation Map Information SEI for reducing energy consumption of displays [O. Le Meur, C.H. Demarty, F. Aumont, E. François, L. Blonde, E. Reinhard (InterDigital)]
• JVET-AD0137 [AHG8] SEI for encoder optimization information [C. Kim, Hendry, D. Gwak, J. Lim(LGE)]
• JVET-AD0161 AHG9: Alternative Output Timing Hint SEI [H.-B. Teo, J. Gao, C.-S. Lim, K. Abe (Panasonic)]
• JVET-AD0175 AHG9: On object mask auxiliary picture and object mask information SEI message for VSEI and HEVC [J. Chen, S. Wang, Y. Ye (Alibaba)]

Activities

The regular JVET e-mail reflector was expected to be used for discussions (jvet@lists.rwth-aachen.de) with [AHG9] in message headers. However, there were no emails sent to the JVET reflector during the AHG period with [AHG9] in the message header.

Recommendations

The AHG recommended to:

• Review all related contributions; and
• Continue SEI messages studies.

A BoG (coordinated by Y.-K. Wang. M. Hannuksela, S. Deshpande) was formed to review the contributions on NNPF SEI messages, and relationship with processing order SEI. The “additional SEI messages” and non-NNPF from the ongoing amendment were to be discussed in main track.