JVET-R0289 [AHG16] On deblocking filter process [N. Park, J. Nam, H. Jang, J. Lim, S. Kim (LGE)]
In Ticket #899, it was claimed that there is a mismatch between VVC draft 8 and VTM8.0 on deblocking filter process when CU size is greater than maximum transform block size and cu_coded_flag is equal to 0.
In VVC draft 8, implicit TU tiling is applied even though cu_coded_flag is equal to 0 for inter prediction mode. These internal transform block boundaries within the CU is considered as transform block boundary in deblocking filter process. However, in the VTM8.0, the above case is treated as a TU which has the same size as CU therefore there is no internal transform block boundary. Although the mismatch has already resolved by the ticket #899, this cause misalignment in terms of filter length on the each block boundary with same property.
In this proposal, to fix abovementioned behavior two solutions are suggested.
- Method1: Implicit TU tiling is restricted when cu_coded_flag is equal to 0.
- Method2: Implicit TU tiling is retained but edges inside a CU are not treated as transform edge when cu_coded_flag is equal to 0.
Since Ticket #899 is reverted by Method 1 and deblocking filter process is fixed to be performed same as VTM8.0 by Method 2, the experimental results for both solutions are same as results of VTM8.0.
In revision 1, it is added the simulation results based on Ticket #899 bugfix.
After the fix of ticket #899, there is no mismatch between text and software.
The suggested solution 1 would revert the solution of the ticket, and require additional checks.
Also the solution 2 would be requiring additional logic.
There is no problem with the current design, no need for action.