JVET-R0133 AHG16: On Clipping values for Non-linear ALF [T. Tsukuba, M. Ikeda, Y. Yagasaki, T. Suzuki (Sony)]

In VVC WD, clipping values for Non-linear ALF filtering process are defined in a lookup table, as a form of “2^N” depending on BitDepth and clipIdx, where N is an integer value. When BitDepth is equal to 16 and clipIdx is equal to 0, clipping values become 2¹⁶; Thus, the lookup table needs up to 17 bits per element.

This contribution proposes to replace the clipping values in a form of “2^N” to a form of “2^N - 1”; Any clipping values are kept within 16 bits and can be derived by simple logical operation without the lookup table. It is asserted that the proposed method has negligible bdrate changes of (0.00%,0.00%, 0.00%) for AI, (0.00%, -0.05%, 0.01%) for RA and (0.01%, -0.06%, -0.17%) for LDB under CTC.

In v2, results of IBDI equal to 8/12 are attached.

It is observed that:

• For IBDI=8, average bdrate changes are (0.02%, 0.10%, 0.11%) for AI, (0.03%, -0.19%, -0.21%) for RA and (0.02%, -0.40%, -0.44%) for LB.
• For IBDI=12, average bdrate changes are (0.00%, -0.01%, 0.01%) for AI, (0.00%, -0.03%, 0.07%) for RA and (0.05%, 0.03%, 0.04%) for LB.

Presented Thu April 9 1705 UTC (chaired by JRO)

The advantage would only apply to case of profiles beyond 15 bit, where the cost of lookup table storage seems almost irrelevant

There was a contribution in Gothenburg (JVET-O0188) which proposed the same approach. It was not adopted by that time

Several experts expressed that this change is not needed, as in the only case that would require 17 bit implementation clipping would have no effect and could be skipped.

No action was taken on this.