JVET-L0359 AHG12: Flexible tile partitioning [Y. Yasugi, T. Ikai (Sharp)]
This contribution is relevant to the tile functionality proposed in JVET-K0155. The proposed tile allows to split a picture into flexible partitioning tiles, where the width or height of the unit of tile can be multiplies of 4 (the minimum CU size), 8, 16, 32 and 64, i.e. smaller than CTU. By using partial CTUs on the right and bottom edges of each tile, a picture can be divided into tiles of more uniform size and this functionality reportedly helps better load balancing. Furthermore, it is also asserted that this feature is useful for 360 video sequences and frame packing sequences since the corresponding tile can fit the arbitrary rectangular face size. The authors implemented the proposed tile on VTM-2.0.1 with support of LFCrossTileBoundaryFlag in adaptive loop filter (ALF) and evaluated the performance.
The experimental results with this feature (unit=32) reportedly show that luma BD-rate coding losses on average are 0.85% for All Intra and 1.45% for Random Access under the common test condition (CTC) for SDR sequences excluding padding data added by CABAC.
The experimental results without this feature (i.e. HEVC like tile) reportedly show that luma BD-rate coding losses on average are 0.82% for All Intra and 1.37% for Random Access under the common test condition (CTC) for SDR sequences excluding padding data added by CABAC.
This basically proposes treating the right edge of a tile as the right edge of a picture for the decoding of each tile, except perhaps for inter prediction references to positions in reference pictures. The positions of CTUs in the picture would not be on a CTU grid anymore. Whenever a new tile would start, a new CTU would start at that position. The number of CTUs in a picture would become somewhat larger. Some constraints on minimum tile sizes might need to be imposed.
An example use case that was discussed was 360° video with face sizes desired not to be a multiple of the CTU size.
This seemed intriguing.
It was noted that if the positions of the tile boundaries change from picture to picture, the positions of CTU boundaries in a picture might not be at CTU boundaries in the reference pictures.
This was further discussed Thursday 11 October 1320 (GJS).
Draft text was provided.
It was noted that an encoder can use smaller CTUs as one way to provide finer tile granularity.
It was commented that if this is motivated by load balancing, it does not seem so interesting for that purpose. However, it was suggested that this can be useful for certain applications such as cubemap coding.
It was suggested that a level limit could be established for how many CTUs (counting partial CTUs as full CTUs) are in a picture rather than the width and height of the picture, or otherwise to limit how many partial CTUs need to be processed).
It was agreed that there were two other aspects that may need to be considered:
- Whether the boundaries are padded when using MVs to reference areas outside the tile (or tile group) in reference pictures. The proposal does not propose padding.
- Whether the CTU positions in a reference picture are at the same place as in the current picture. This may not be an issue, as long as the boundaries are on an 8x8 basis (due to storage of MV data for TMVP on an 8x8 basis).
This seems conceptually OK, and likely to be useful for some applications. It was commented that the software looks good.
The group was inclined to adopt this, but planned to wait one meeting cycle for study to confirm that no problems would be identified.