JVET-K0230 CE1-related: Separate tree partitioning at 64x64-luma/32x32-chroma unit level [T.-D. Chuang, C.-Y. Chen, Y.-W. Huang, S.-M. Lei (MediaTek)]

In a typical pipelined hardware decoder architecture, the data is pipelined with NxN luma blocks and MxM chroma blocks, where NxN and MxM are the same as the maximum luma transform block (TB) size and the maximum chroma TB size. In VTM-1.0, the maximum luma and chroma TB sizes are 64x64 and 32x32. In separate tree partitioning, for each coding tree unit (CTU), the luma coding tree block (CTB) is first signalled, then the chroma CTBs are signalled, which makes hardware decoder architecture to change from maximum TB pipelining to CTU pipelining (i.e., the data is pipelined with luma CTBs and chroma CTBs). In VTM-1.0, the luma CTB size is 128x128, and the chroma CTB size is 64x64. The number of samples for a decoder pipeline stage to support CTU-level separate tree partitioning is four times of that without separate tree partitioning, which will lead to significant increase of hardware areas for pipeline stages. In this contribution, it is proposed to start separate tree partitioning starting at 64x64-luma/32x32-chroma units instead of CTU. In the proposed method, each intra-slice CTU is first implicitly split into 64x64-luma/32x32-chroma units. Then the coding tree under each 64x64-luma/32x32-chroma unit is separated, and luma syntax is signalled before chroma syntax within each 64x64-luma/32x32-chroma unit. It is claimed that the proposed method can be easily supported by hardware decoder architecture of data pipelining with 64x64 luma blocks and 32x32 chroma blocks. Compared against the CTU-level separate tree partitioning, simulation results reportedly show negligible BD-rate differences for the 64x64-luma/32x32-chroma unit-level separate tree partitioning. It is also reported that the 64x64-luma/32x32-chroma unit-level separate tree partitioning with multiple intra chroma direct modes (multi-DMs) can achieve 0.99% and 0.81% Y BD-rates, 11.24% and 9.80% U BD-rates, 11.20% and 9.80% V BD-rates, 17% and 40% encoding time decreases for VTM-1.0-AI and BMS-1.0-AI, respectively, when both the anchor and the test include linear model (LM) chroma mode and the 65 intra angular modes. The impact on decoding time seems negligible.

It was commented that the current shared tree scheme also is not friendly to 64x64 pipeline architecture if there is a top-level split that is a ternary split.

It was suggested that we could just have a smaller maximum CU size for intra than inter. The contributor said this would likely have no impact on coding efficiency.

It was noted that there is an interaction with CCLM.

For separate tree operation, the contribution proposed that all luma would be sent before all chroma on a 64x64 basis or a CTU basis, whichever is smaller.

Decision: Adopt separate trees for intra slices (without multiple intra chroma direct modes) with an implicit split to 64x64 (into both VTM and BMS).

Decision: Prohibit ternary split of something bigger than 64 in width or height (and not send the bit to indicate ternary type at that level). See also later contribution JVET-K0556 and the notes of the plenary discussion in section 11.2, in which it was agreed that this prohibition affects both intra and inter.

As a change of the software and CTC configuration, it was suggested to increase the chroma QP for intra when the trees are separate. The contributor had tested increasing the chroma QP offset by 1 for intra (with CCLM in both the anchor and test) and said this showed an increase to 3%/1.5%/0.4% over the VTM, that they would provide the test results in a revision of the contribution. Decision (SW & CTC): Agreed.

There was further discussion on Monday 1500 (GJS) about why separate trees are only planned for intra slices rather than also intra CTUs in inter slices. It was commented that the intra/inter switch is at the CU level rather than the CTU level, so in the current scheme the tree ends before the intra/inter decision is made. A proposed approach (JVET-K0354) that was studied in CE1 was to add a flag at the CU level that would continue the tree, and the two trees would separate from that point downward. It was remarked that the way current-picture referencing works means that if we only support separate trees for intra slices, we cannot combine separate tree usage with current picture referencing.

This was further discussed Tuesday 0945 (GJS). It was noted that supporting separate trees for intra CUs in inter slices shows little improvement in the CTC and has an encoder complexity impact, but it was reported that there was little decoder complexity impact, and enabling this would make intra more consistent between inter slices and intra slices (e.g., an encoder would not need to change the slice type in order to get access to the separate tree functionality, and some encoders may seldom use intra slices). There was some questioning of decoder complexity impact, but there was no clearly significant impact on decoder complexity).

In Track A, it was initially agreed to enable separate trees for intra CUs in inter slices (relating to JVET-K0354).

Decision: There should be a high-level (e.g., SPS) flag to enable or disable the use of separate trees (for intra CUs in intra slices).

For intra CUs in inter slices, it was agreed that if the use of separate trees was enabled, there should be another flag to enable it or disable it in inter slices. The use in inter slices was to be disabled in the CTC. However, since experiments appeared to show some complexity impact on the decoder, further study in a CE was planned to be performed to confirm the decoder complexity impact before taking action on enabling separate trees for intra CTUs or CUs in inter slices.

This initial agreement was further discussed in JVET plenary Tuesday 1330 (GJS & JRO). It was suggested to consider a different approach for inter slices – which is that, in an inter slice, a flag at the whole-CTU level would indicate that the CTU is intra with separate trees. If the flag is zero, there would be the current scheme with a common tree. It was agreed that this and the other described scheme should be tested in a CE.

It was also suggested to consider a local switch at the CTU or CU level for whether a separate tree or a single tree or common tree is used. Such a local switch could be used in intra slices as well as in inter slices. This should also be tested in the CE.

As a matter of design principle, it was agreed that having intra work similarly in an inter slice as in an intra slice is desirable (in the absence of some justification to do otherwise). However, no action was taken to choose a particular method of enabling separate trees in inter slices for current inclusion in the VTM or BMS.