JVET-W0075 Subpicture-wise reference picture resampling and scalability for Game/E-sport contents streaming [B. Choi, S. Wenger, X. Li, S. Liu (Tencent)]
For game/e-sports content streaming use cases including:
- Local enhancement of visual quality or resolution
- Customized layout of the played video
- Update of layout within a coded video sequence,
The contribution proposes the following modifications of VVC high-level syntax:
- Signalling a scaling window per subpicture, when more than one subpicture is present: For subpicture-wise reference picture resampling in a layer or across layers, scaling windows need to be signalled in PPS for each subpicture that has a different scaling ratio with the reference subpicture.
- Signalling the updated subpicture partitioning information in PPS: To enable the picture-level updates of subpicture partitioning information, the subpicture partitioning information needs to be signalled in PPS, optionally. When the subpicture partitioning information is updated, some subpictures with particular subpicture IDs may appear or disappear, and the spatial resolution and the vertical and/or horizontal offset values of each subpicture may be changed.
- Permitting that each layer may have a different subpicture partitioning layout.: In VVC version 1, all layers belonging to the same dependency tree shall have the same subpicture partitioning layout. In VVC next versions, it is proposed to relax this constraint to support the argued use cases that need different subpicture layouts across layers.
- Permitting the combination of subpicture partitioning and reference picture resampling: In VVC version 1, the combination of subpicture partitioning and reference picture resampling is not supported. We propose to relax this constraint to support region-wise quality/resolution enhancement.
Many of these aspects might fall into areas which could better be handled at systems level (see discussion under JVET-W0169). The approach of RPR/scalability at subpicture level would definitely require changes in VVC (it was by purpose left out of V1 due to complexity consideration). However, evidence would be required that this provides benefit e.g. in terms of compression performance, and that the additional complexity would be justified.
It is further commented that this is not purely high-level syntax, but rather functionality with non-negligible impact on low-level implementation.
Further study was recommended.