JVET-L0083 CE2-related: Reduction of bits for ALF coefficient fractional part [Y.-C. Su, C.-Y. Chen, Y.-W. Huang, S.-M. Lei (MediaTek)]
In the contribution, it is proposed to reduce bits for adaptive loop filter (ALF) coefficient fractional part. In VTM2.0.1, the fractional part of each ALF coefficient is represented in nine bits. It is reported that, compared with VTM2.0.1, 6-bit fractional part results in 0.10%, 0.09%, and -0.04% luma BD-rates for AI, RA, and LB, respectively, and 7-bit fractional part results in -0.01%, -0.03%, and -0.09% luma BD-rates for AI, RA, and LB, respectively. The reduction of bits for ALF coefficient fractional part is also tested on top of CE2.2.2 from JVET-L0082. The real value range of the center ALF coefficient is within [0.0, 2.0), the real value range of non-center ALF coefficients is within [-1.0, 1.0), and the number of bits for ALF coefficient fractional part is six or seven. It is shown that compared to VTM2.0.1, 6-bit fractional part with the reduced ranges results in 0.10%, 0.08%, and -0.03% luma BD-rates for AI, RA, and LB, respectively, and 7-bit fractional part with the reduced ranges results in -0.01%, -0.04%, and -0.11% luma BD-rates for AI, RA, and LB, respectively. With the 7-bit fractional part with the reduced ranges for ALF coefficients, the numbers of required bits to represent a non-center coefficient and a center coefficient are reduced from 11 to 8 and 15 to 8, respectively, when compared to VTM2.0.1. There are almost no run time changes in all the above tests.
This doesn’t change how the data is coded – only reduces the number of fractional bits for ALF coefficients from 9 to 7. This would enable use of an 8-b multiplier instead of a higher-precision multiplier. This a refinement relative to the CE2.2.2.
It was not clear how this could produce gain.
Decision: Adopt (text is in the contribution).