JVET-P0803 Non-CE3: Combined cleanup of MIP [J. Pfaff, B. Stallenberger, M. Schäfer, P. Merkle, T. Hinz, P. Helle, H. Schwarz, D. Marpe, T. Wiegand (HHI), T. Biatek, A. K. Ramasubramonian, G. Van der Auwera, M. Karczewicz (Qualcomm), H. Liu, L. Zhang, K. Zhang, Z. Deng, J. Xu (Bytedance), J. Choi, J. Heo, J. Lim, S. Kim (LGE), K. Naser, T. Poirier, F. Le Leannec, F. Galpin (Inter Digital)] [late]
This joint proposal combines several proposals on matrix based intra prediction (MIP) that were submitted to the 16th JVET Meeting in Geneva. It proposes to adopt technologies that were proposed in the following documents: JVET-P0054, JVET-P0194, JVET-P0352, JVET-P0398 and JVET-P0535.
The BD-rate performance of the proposed combination compared to VTM-6.0 is -0.06% / -0.1% / -0.07% (101% enc. / 101% dec.) for AI configuration and -0.03 / 0.00% / -0.01% (101% enc. / 103% dec.) for RA configuration without the additional context and -0.08% / -0.11% / -0.13% (102% enc. / 102% dec.) for AI configuration and -0.04%/-0.04%/-0.12% (102% enc. /102% dec.) for RA with the additional context.
Proposed aspects:
- MIP is implemented with a fixed upsampling order as the test CE3-3.2 in JVET-P0054 (upsampling is first performed horizontally, then vertically).
- All MIP-modes are allowed to be transposed and a separate transposed flag is signalled as in JVET-P0194 (only described aspect).
- MIP is allowed up to 64x64 regardless of the maximum transform size, where TU tiling is performed for larger CUs as in JVET-P0352 and JVET-P0198.
- The number of MIP modes is 32 for sizeId=0, 16 for sizeId=1 and 12 for sizeId=2 as in JVET-P0398.
- The restriction of the usage on MIP to certain block shapes is removed as in JVET-P0535.
- A separate context for the first bin of the MTS-index depending on MIP is used as in JVET-P0398 (optional).
Aspect 2: it is questioned that this is a clean-up. It is suggested by proponent and non-proponent that it is a clean-up by removing special case where transposing is not allowed.
Aspect 3: commented that the gain is minor, however, it is claimed to be a fix for this non-CTC case when the max transform size is set to 32x32
Aspect 5: does it bring gain? It is claimed that there is some gain. Restriction is removed. Matrices were retrained. JVET-P0535 reports results with current matrices, however, JVET-P0398 reports results with retrained matrices (combines aspects 4, 5, 6).
It was commented that removing restrictions on block sizes and aspect ratios results in additional validation testing, while coding gain is unclear.
It was commented that removing the conditions are reasonable by non-proponent. The benefit is that there are fewer conditions in the syntax.
The proponent comments that the runtime measurement is inaccurate.
It was commented that not all aspects may be clean-ups in the combined proposal.
Separate results are suggested to be reported for the aspects:
- Non-CTC content: combined proposal result using Ericsson data set
- Aspects 2, 4, 5: separate result available in JVET-P0398 to be reported
- Aspect 3: non-CTC conditions with max transform size restriction are reported JVET-P0352 and JVET-P0198
- Aspect 5: separate result available in JVET-P0535 to be reported
- Aspect 6: separate result available in JVET-P0398 to be reported
Additional information is requested on additional block sizes that are included (what is currently in draft spec and what are additional shapes and it is requested to explain benefits of additional shapes), binarization.
The BoG recommended further discussion in track A.