JVET-P0613 Non-CE5: On the average calculation for luma adaptive deblocking filter [B. Ray, D. Rusanovskyy, G. Van der Auwera, M. Karczewicz (Qualcomm)]

The contribution proposes to modify the average luma calculation value for luma average dependent deblocking filter, by utilizing the information of maximum filter length from side P and side Q. The results for SDR-CTC and HDR-CTC conditions are reported below.

In the current VTM, the nearest samples to the block boundary are used for the average luma calculation. It is proposed to use the average luma calculation based on the maximum filter size for the block.

In the current VVC, the calculations for the average is

LL= ( ( p_0,0 + p_0,3 + q_0,0 + q_0,3 ) >> 2 ) / ( 1 << iBitDepth ).

It is proposed to replace it with

LL= ( ( p_0,0 + p_maxP,0 + p_0,3 + p_maxP,3 + q_0,0 + q_maxQ,0 + q_0,3 + q_maxQ,3) >> 3 ) / ( 1 << iBitDepth ),

A proponent of the initial average luma deblocking filter has expressed support.

We optimal parameters for the average luma DF for the proposal to show benefit of the proposal.

BoG recommendation: Further study.