JVET-Q0157 AHG12: On motion compensation for sub-pictures [V. Seregin, Y.-J. Chang, M. Coban, M. Karczewicz (Qualcomm)]
This contribution was discussed Tuesday 7 January 2020 at 1845 (chaired by GJS).
- To fix an asserted decoding problem of repositioned sub-pictures by considering boundaries of a reference sub-picture in the motion compensation process and TMVP derivation.
In the current text, inter prediction does not check the subpicture ID of the collocated region in the reference picture.
It was commented that it is intended that when the subpicture ID of a position in the picture changes in bitstream order, the slices of the subpicture would need to be IRAP slices, although this is not currently expressed in the text.
The proposal is to offset the position in the reference picture by the difference in position of the subpicture with the same subpicture ID. (A constraint would be needed that the reference picture must contain a subpicture with the same ID.)
It was commented that we should also think about whether the subpicture would have the same size in the reference picture.
It was commented that the proposed scheme could avoid the need for some post-decoding repositioning of the decoded regions – for example, the decoded picture could be made more spatially sensible, rather than using some post-decoding operation to rearrange the regions into a spatially sensible picture. However, another participant indicated that post-decoding rendering work is ordinarily needed anyway.
This would have an offset per subpicture for the motion compensation position calculation, which has some low-level decoding process change. The range of values of MVs could become larger than what it would otherwise be unless there is some additional constraint.
Temporal MVP would also involve considering the spatial offset relative to the corresponding position in the collocated picture.
While some participants liked the idea, we need to be very conservative about introducing changes to the low-level decoding process.
Further study of potential decoding process change was encouraged.
Text for the IRAP constraint was prepared by YKW.
This was further discussed Thursday 16 january in Track A at 1320 (chaired by GJS)
Decision (expression of existing intent): “When the current picture is not the first picture of the CLVS, for each value of i in the range of 0 to sps_num_subpics_minus1, inclusive, if the value of SubpicId[ i ] is not equal to the value of SubpicId[ i ] of previous picture in decoding order in the same layer, the nal_unit_type for all coded slice NAL units of the the subpicture with subpicture index i shall be in the range of IDR_W_RADL to CRA_NUT, inclusive.”
- Additionally, it is proposed to replace a current picture size with a reference picture size in the clipping (motion padding) since the reference picture may have different size than the current picture when reference picture resampling is enabled.
Decision (bug fix of existing intent): Adopt this aspect.