JVET-C0061 Decoder-side intra mode derivation [X. Xiu, Y. He, Y. Ye (InterDigital)]
In this contribution, a decoder-side intra mode derivation (DIMD) approach is proposed based on the JEM. The DIMD approach derives the intra prediction mode and is asserted to reduce the overhead of intra mode signalling. It is asserted that because the only mode signalling needed in DIMD is the indication to use the DIMD mode and the level at which the prediction operates, intra prediction can be performed at higher granularity than in the JEM. For an intra CU with 2Nx2N partition, one DIMD flag is signalled. When the flag is true, the intra mode of the CU is derived using a template-based method at both encoder and decoder. Additionally, when the DIMD mode is selected, another control flag is signalled to indicate whether the intra mode is derived at PU-level or TU-level. For the intra CU with NxN partition, the intra modes derived by DIMD are used to replace one exiting MPM candidate for the intra mode coding of four PUs in this CU. It is reported that, compared to the JEM-2.0 anchor, the proposed method achieves on average 0.81%, 0.47%, 0.21% and 0.23% luma BD-rate reductions for AI, RA, LDB and LDP configurations, respectively; when compared to the anchor by disabling all JEM tools, the corresponding average BD-rate reductions are reported as 1.37%, 0.72%, 0.27% and 0.40%.
A one-line-wide reference is used to determine an intra prediction direction for predicting using an L-shaped template which has a width of 2 or 4 samples, depending on the block size.
A fast search is used; however, the encoder and decoder runtime is still increased by roughly 50%. In the worst case, 20 modes have to be tested.
DIMD, in the case of 2Nx2N is signalled by an additional flag, no MPM is used, and another additional flag is also added that indicates whether it is invoked at the TU or PU level. For NxN, DIMD is put in the first place of the MPM list. A question was raised why this is not done equal? It is reported that it would have some loss.
The technology gives interesting gain, but increases decoder (and encoder) complexity significantly. The main aspect would be reducing the number of searches at the decoder side, to achieve a better tradeoff of complexity vs. compression benefit (and still retain the gain in compression).
Further investigate in EE was planned, with the following aspects to be investigated:
- limitation of number of lines used in the search (1, 2, 4)
- restriction of not using it for small blocks
- restricting number of candidates further
- removing parsing dependencies (so it shall not rely on reconstructed samples)
- SIMD for the decoder (but should be same for intra and inter)
- align with QTBT; if it is intended to still retain two different ways of treating for larger and smaller blocks (as currently for 2Nx2N n NxN), the benefit of this should also be reported
- comparison against an anchor with similar encoder complexity (using other than the fast search for intra mode), and also identifying whether decoder-side derivation still gives similar gains when the encoder would check more modes