JVET-AE0053 AHG8/AHG9: Neural-network post-filter regions SEI message [T. Chujoh, Y. Yasugi, T. Ikai (Sharp)]

JVET-AE0053 AHG8/AHG9: Neural-network post-filter regions SEI message [T. Chujoh, Y. Yasugi, T. Ikai (Sharp)]

This was discussed in JVET on Friday July 14.

The neural-network post-filter characteristic and activation SEI messages process picture by picture. In several past meetings, there were several proposals to activate regions in the picture, and it has been recommended to use the annotated regions SEI message, which carries parameters that identify annotated regions using bounding boxes representing the size and location of identified objects. However, information on using regions and labels specified in the annotated regions SEI message for post-processing is not defined. Therefore, this contribution considers a minimum specification of how to post-process regions and introduces a new SEI message indicating an ID for the annotated regions SEI message. This proposal is expected to apply to the VSEI (V4) and is asserted to be useful not only for ROI but also encoders and receiving systems for machine analysis of coded video content.

It was commented that region based activation of NNPF is a useful functionality. However, the combination with annotated regions SEI message may not be the best way to implement this, as its original purpose is different (more in the semantics of content, rather than processing). Furthermore, such an SEI message should support multiple regions, and also using different NNPF for different regions.

The functionality of region based processing is not supported in v3, and would be interesting for a future version. Further study was recommended.