JVET-R0208 AHG16: Rounding correction for ALF virtual boundary processing [A. M. Kotra, S. Esenlik, B. Wang, H. Gao, E. Alshina (Huawei)]
In VTM-8.0, to avoid extreme padding for the sample rows which are immediately adjacent to the adaptive loop filter (ALF) virtual boundary, the correction value applied during the filtering is quantized by a larger value 1024 (210) instead of 128(27). However the rounding value used during the filtering is still 64. The current proposal proposes a fix by changing the rounding value to 512 when the quantization value used in the ALF filtering is 1024.
The objective results, over VTM8.0 Anchor for CTC configuration are as follows:
Config. Y U V EncT DecT
AI 0.00% 0.01% 0.01% 100% 99%
RA 0.01% -0.01% 0.00% 100% 100%
LDB 0.01% 0.02% -0.07% 100% 102%
LDP 0.01% -0.25% -0.14% 100% 103%
By modifying filters at virtual boundaries per adoption of JVET-Q0150 solution 2, the rounding operation in case of shift 10 is no longer doing the nearest integer rounding. This appears as an inconsistency rather than a bug. This issue had been detected during software integration. Proponents of JVET-Q0150 also support this change. It is agreed that the change is minor and there is no harm that it would introduce any problems.
JVET-R0231 method 1 and JVET-R0291 target the same problem, basically the same solution but different specification text. There may be more elegant ways of expressing the change than suggested in JVET-R0208, which introduces another column in the table 45/46, e.g. by an equation.
The AHG meeting recommended that the rounding operation in case of the modified filter at virtual boundary should be aligned, and that editors should decide the best way of expressing it in text.
Was confirmed in track B Tue 21 Apr. 21
Decision (cleanup/text+software): Adopt JVET-R0208. The rounding operation in case of the modified filter at virtual boundary should be aligned. Editors should decide the best way of expressing it in the spec. text.