JVET-AK0197 AHG13: on picture size for the interpretation of the FGC SEI message [P. de Lagrange, E. François (InterDigital)]

This contribution proposes to amend the VVC specification to refer to SPS instead of PPS to define picture resolution for the interpretation of the film grain characteristics SEI message, the intent being to describe film grain characteristics at source picture resolution independently of encoder decisions to perform reduced-resolution coding.

It is also proposed to indicate that unspecified cropping and padding can be applied as needed.

Upsampling the video with film grain synthesized at low resolution seems not appropriate. Therefore this modification clarifying how the SEI message is to be used together with RPR appears reasonable.

This was further discussed in plenary on Sunday. It is agreed that the problem exists, as the picture width/height can vary in a sequence at picture level due to encoder optimization, where just an off-the-shelf encoder might be operated which is not specifically knowing about film grain. It was however commented that the problem requires further study and the proposed solution might potentially violate some of the principles applied so far in interfacing VVC and VSEI. It was furthermore commented that the impact on existing hardware implementations should be studied. A simple solution would be adding a note in VSEI that it is not recommended to use FGC when RPR is used in VVC.

In a follow-up discussion on Sunday morning, it was commented by GJS that when originally designing the text in VVC on film grain, it was not expected that it was used with RPR. The concept of assuming that the max resolution signalled in SPS would be the one to be used, is not following the usual assumption that the SPS would express the original picture size, and that the max parameters are really indicating what happens in the bitstream. These are rather foreseen to indicate the needs of max buffer size. Furthermore, the proposal might break existing implementations of the SEI message.

A potential solution might be to express the case “when the pic width/height changes within a CVS”, where it would be up to the implementer of the SEI message which of the values to be interpreted as PicWidth/PicHeight in the SEI message.

It was also mentioned that at least two sequence-level flags exist (e.g., sps_ref_pic_resampling_enabled_flag) that signal potential usage of RPR, which could be used for this purpose.

Further offline work was requested to be conducted by GJS, PdL, YKW, WH, JS.

It was reported that an idea of resolving the issue in a way that is in better harmony with the intended usage of SPS was discussed, but then it was found that additional issues might exist when FGC is operating in SPO and resolution is changed. Potentially a new SEI message about intended display size could resolve this issue at least for systems that are able to interpret it.

It was commented that the text considered would improve the situation in the non-SPO case, such that the interpretation of the pic width/height could be handled by a formulation “UpsamplingInferredFlag may be set as follows”, and also restricting this to the case where FGC is not part of SPO, or is the first step would at least improve the situation. It was requested to produce a new input document (see further notes under JVET-AK0339).