JVET-AP0226 AHG9/AHG18: Missed Picture Generation purpose for NNPF SEI message [K. Malyshev, M. Sychev, S. Ikonin, E. Alshina (Huawei)]

This contribution proposes further extension of NNPF SEI messages to support a new purpose for applying NNPF specifically for missed picture generation. The proposal introduces a new MissedGenerationFlag within the nnpfc_purpose syntax element, allowing to dynamically select missed picture handling method. This addition includes the following aspects:

1. A new bit in nnpfc_purpose is dedicated for missed picture generation.
2. When nnpfc_purpose indicates missed picture generation, nnpfc_missed_generation_mode specifies the method, with mode 0 relying on closest picture replacement and mode 1 choosing between interpolation (picture rate upscaling) or temporal extrapolation.
3. A new flag nnpfc_absent_generation_flag for picture rate upsampling NNPF, which removes restrictions to allow generating absent input pictures
4. A new signalling for temporal extrapolation NNPF, that clarifies temporal positions of the extrapolated pictures, eliminating uncertainty.

This contribution signals recommended post-processing that a decoder would apply when the decoder detects a missing picture, e.g. using NNPF for error concealment.

Previously NNPF operation is encoder driven via the NNPFA SEI message. This proposed a fundamental change makes the operation become decoder driven, based on missing pictures.

For the absent generation flag, the proposed language says “the NNPF shall generated at least one new picture…”, which is problematic language for an SEI message.

It was asked if the NNPFC SEI message would be lost when the picture was lost. questioned if an NNPFA SEI message is sent. The proponents suggested that it is possible for the NNPFA SEI message to be inserted by the decoder.

It was asked if the generated picture would be for post-processing, or if it would also impact the DPB? Proponents say it is only for post-processing.

It was questioned if NNPF is the appropriate framework for this.

Proposal 3 proposes more explicit positioning of temporal position of extrapolation.proposed syntax for proposal 3 is not structured in a backward compatible manner.

Additional justification of the use case that requires this additional functionality would be useful.