JVET-P0409 CE1-related: Enable PROF for RPR [J. Chen, R.-L. Liao, J. Luo, Y. Ye (Alibaba)]
At the 15th JVET meeting in Gothenburg, reference picture resampling (RPR) (JVET-O1164) is adopted to support adaptive resolution change (ARC) in VVC. Since the existing process of prediction refinement with optical flow (PROF) does not support the case when the reference picture is of different spatial resolution with the current picture, PROF is disabled for this case in VTM-6.1. In this contribution, two alternative methods are proposed to enable PROF to support RPR.
It is reported that by enabling PROF in CE1 anchor with the proposed method 1 and method 2, the coding performance is as follows:
- Method 1 & 3
- 2:1 ratio: {-0.09%, -0.05%, -0.15%} for PSNR1 and {-0.07%, -0.04%, -0.11%} for PSNR2
- 1.5:1 ratio: {-0.04%, 0.20 %, 0.01%} for PSNR1 and {-0.02%, 0.17%, 0.00%} for PSNR2
- Method 2
- 2:1 ratio: {-0.11%, -0.07%, -0.09%} for PSNR1 and {-0.08%, -0.08%, -0.07%} for PSNR2
- 1.5:1 ratio: {-0.05%, 0.19%, 0.16%} for PSNR1 and {-0.02%, 0.14%, 0.17%} for PSNR2
It was commented that the intent was for PROF to be disabled. (PROF and RPR were adopted at the same meeting.)
"Method 1" and "Method 3" are effectively the same, since the only difference between them is for a case that is not allowed in the draft standard.
It was commented that the process involves accessing an extra sample around the prediction block, and getting that extra sample involves performing resampling. It would therefore have higher complexity than if the combination is prohibited. An analogous case exists for BDOF, and the combination was prohibited because of this. The extra fetch would not be not be needed with "Method 2", but it would require a change to how PROF works, in a way that would not otherwise be desirable.
It was commented that using RPR is not expected to be applied on every frame.
The benefit did not seem to justify the implementation difficulty.
Decision: Prohibit the combination.