JVET-M0031 CE11: Summary Report on Deblocking [A. Norkin, A. M. Kotra]
This contribution provides a summary report of Core Experiment 11 on deblocking filtering. Two categories of proposals are covered by this CE, split into two sub-tests. These sub-tests are 1) long-tap deblocking filters, and 2) deblocking at 4x4 block boundaries.
The proposals were encoded according to two test conditions, which correspond to two anchors.
Anchor-1 is VTM-3.0 according to the CTC.
Anchor-2 is VTM-3.0 with ALF switched off (other conditions are the same as in Anchor 2).
Test# | Description | Document# |
11.1.1 | Very strong deblocking filtering with conditional activation signalling | |
11.1.2 | Extended deblocking filter for luma and chroma | |
11.1.3 | Long deblocking filters (only luma) | |
11.1.4 | CE11: Test results of CE11.1.5 long-tap deblocking filter | |
11.1.5 | Longer tap Luma deblocking filter | |
11.1.6 | Combination of JVET-L0403, JVET-L0327, JVET-L0405, JVET-L0140. JVET-L0337, JVET-L0572, JVET-L0072 | |
11.1.7 | Combination of JVET-L0403, JVET-L0327, JVET-L0405, JVET-L0140. JVET-L0337, JVET-L0572, JVET-L0072 with 6 line buffers at CTU boundary | |
11.1.8 | Combination of JVET-L0403, JVET-L0327, JVET-L0405, JVET-L0140. JVET-L0337, JVET-L0572, JVET-L0072 with 4 line buffers at CTU boundary | |
11.2.1 | Deblocking for 4xN, Nx4 blocks and 8xN, Nx8 blocks that are not aligned with 8x8 sample grid | |
11.2.2 | Parallel deblocking filter |
Test CE11.1
Tests | Luma modified (Y/N) | Chroma modified (Y/N) |
CE11.1.1 | Y | Y |
CE11.1.2 | Y | Y |
CE11.1.3 | Y | N |
CE11.1.4 | Y | N |
CE11.1.5 | Y | N |
CE11.1.6 | Y | Y |
CE11.1.7 | Y | Y |
CE11.1.8 | Y | Y |
Luma deblocking complexity
Tests | Samples from block bound. modified | Samples from block bound. deblocking decision | Max num. oper for filtering per line (add/mult/compar/shift) | Max number of oper. for decision per line (add/mult/compar/shift) | Num. line buffers | Worst case complexity increased (Y/N) |
VTM3.0 | 3+3 | 4+4 | 56 (28/2/12/14) | 94 per 8 line segment 11.75 per line | 4 | |
CE11.1.1 | M=3+3...16+16 | 1+1 | (2/3/2/1)*M | 6 (5/0/0/1) per line | 4 | N |
CE11.1.2 | 7+7 | 8+8 | 246 (138/12/28/68) | 4.5 (12/0/6/0)/4 in additional to VTM | 8 | Y (for filtering) |
CE11.1.3 | 7+7, 7+3, 3+7, 3+3 | 16 | 266 (168/34/28/36) | 12.5 (24/0/22/4)/4 in addition to VTM | 8 | N |
CE11.1.4 | 7+7, 3+7 | 16 | 190 (124/12/0/54) | 2.5 (6/0/2/2)/4 in addition to VTM | 4 | N |
CE11.1.5 | 7+7, 3+7 | 16 | 187(85/4/28/42) | 9 (24/4/4/4)/4 in addition to VTM | 4 | N |
CE11.1.6 | 7+7, 7+5, 5+7, 5+5, 7+3, 3+7, 5+3, 3+5 | 8+8, 8+6, 6+8, 6+6, 8+4, 4+8, 6+4, 4+6 | 148 (74,24,28,22) | 12 (26/0/10/12)/4 in addition to VTM | 8 | N |
CE11.1.7 | 7+7, 7+5, 5+7, 5+5, 7+3, 3+7, 5+3, 3+5 | 8+8, 8+6, 6+8, 6+6, 8+4, 4+8, 6+4, 4+6 | 148 (74,24,28,22) | 12 (26/0/10/12)/4 in addition to VTM | 6 | N |
CE11.1.8 | 7+7, 7+5, 5+7, 5+5, 7+3, 3+7, 5+3, 3+5 | 8+8, 8+6, 6+8, 6+6, 8+4, 4+8, 6+4, 4+6 | 148 (74,24,28,22) | 12 (26/0/10/12)/4 in addition to VTM | 4 | N |
Chroma deblocking complexity
Tests
| Samples from block boundary modified | Samples from block boundary for deblocking decision | Max number of operations for filtering per line (add/mult/compar/shift) | Max number of oper. for decision per line (add/mult/compar/shift) | Number of line buffers | Worst case complexity increased (Y/N) |
VTM3.0 | 1+1 | None | 10 (6/0/2/2) | None | 2 | |
CE11.1.1 | M=1+1...8+8 | 1+1 | (2/3/2/1)*M | 6 (5/0/0/1) per line | 2 | Y |
CE11.1.2 | 3+3 | 4+4 | 56 (28/2/12/14) | 17.5 (19/0/10/6)/2 per line | 4 | Y |
CE11.1.3 | VTM | VTM | VTM | VTM | 2 | N |
CE11.1.4 | VTM | VTM | VTM | VTM | 2 | N |
CE11.1.5 | VTM | VTM | VTM | VTM | 2 | N |
CE11.1.6 | 3+3 | 4+4 | 76 (48, 2, 12, 14) | 17 (17, 0, 12, 5)/2 per line | 4 | Y |
CE11.1.7 | 3+3 | 4+4 | 76 (48, 2, 12, 14) | 17 (17, 0, 12, 5)/2 per line | 3 | Y |
CE11.1.8 | 3+3 | 4+4 | 76 (48, 2, 12, 14) | 17 (17, 0, 12, 5)/2 per line | 2 | Y |
11.1.6-8 are the same (combined from 4 previous proposals) with different mumber of line buffers (8/6/4) at CTU boundary.
11.1.5, 11.1.6-8 use sample-based adaptation of clipping range
It was suggested to translate the maximum number of operations per line into the maximum number of operations per sample (also considering that the long filters are only applied for large blocks), and decode the value “M” of 11.1.1 into a number. Some of the statements in the table above (worst case increased Y/N) may not be true. More detailed analysis is needed but it appears that if proposals increase worst case complexity in terms of number of operations it would not be by a large factor.
This analysis became available in JVET-M0031v4 and was presented in Track B at Friday 18 January 0800 (chaired by JRO).
Beyond the number of operations, the number of line buffers is another complexity issue. VTM uses 4 lines for luma and 2 lines for chroma, which is duplicated by some proposals.
Parallel processing capability is given by all proposals at some sufficiently small granularity (though larger than current VVC) – see table below.
Tests | Smallest unit size needed to perform proposed deblocking operations separately from any other units |
VTM3.0 | 8x8 |
CE11.1.1 | luma 16x16, chroma 8x8 (to be verified) |
CE11.1.2 | 16x16 |
CE11.1.3 | 16x16 |
CE11.1.4 | 16x16 |
CE11.1.5 | 16x16 |
CE11.1.6 | 32x32 |
CE11.1.7 | 32x32 |
CE11.1.8 | 32x32 |
It is however pointed out that some of the proposals 11.1.1-5 did not consider that VTM3 now uses subblock boundary deblocking, whereas the decision of using long filters is based only on CU boundary properties. Therefore, it could happen that a long filter is applied first, and afterwards a subblock boundary within the CU is deblocked again. This would inhibit parallelism. This could be classified as a bug, which however in terms of subjective testing might not be too harmful.
None of the proposals comes with a specification text. For the decision on using long filters, CU and subblock boundaries should be handled equal.
SNR based Results with ALF on
AI | RA | |||||||||
Test | Y | U | V | EncT | DecT | Y | U | V | EncT | DecT |
CE11.1.1 | 0.05% | 0.01% | 0.03% | 100% | 100% | 0.35% | 0.12% | 0.12% | 100% | 101% |
CE11.1.2 | -0.01% | -0.56% | -0.52% | 100% | 102% | -0.06% | -0.41% | -0.32% | 100% | 102% |
CE11.1.3 | 0.00% | 0.00% | 0.00% | 100% | 101% | -0.02% | -0.02% | 0.00% | 100% | 103% |
CE11.1.4 | 0.01% | 0.00% | -0.01% | 102% | 103% | 0.00% | 0.01% | 0.00% | 102% | 102% |
CE11.1.5 | 0.00% | 0.00% | 0.00% | 99% | 100% | 0.02% | -0.01% | 0.00% | 99% | 99% |
CE11.1.6 | -0.02% | -0.53% | -0.49% | 100% | 99% | 0.00% | -0.80% | -0.82% | 100% | 102% |
CE11.1.7 | -0.02% | -0.54% | -0.50% | 101% | 99% | -0.01% | -0.82% | -0.85% | 101% | 102% |
CE11.1.8 | -0.02% | -0.43% | -0.39% | 101% | 103% | 0.00% | -0.71% | -0.68% | 100% | 101% |
LD-B | LD-P | |||||||||
Test | Y | U | V | EncT | DecT | Y | U | V | EncT | DecT |
CE11.1.1 | 0.40% | 0.20% | 0.15% | 101% | 102% | 0.40% | 0.22% | 0.05% | 101% | 102% |
CE11.1.2 | 0.03% | 0.53% | 0.23% | 100% | 102% | -0.01% | 0.15% | -0.32% | 100% | 102% |
CE11.1.3 | 0.01% | -0.02% | 0.00% | 100% | 103% | -0.02% | 0.13% | 0.03% | 100% | 102% |
CE11.1.4 | 0.13% | -0.03% | -0.09% | 99% | 103% | 0.06% | 0.21% | -0.10% | 101% | 104% |
CE11.1.5 | 0.13% | 0.15% | 0.07% | 98% | 99% | 0.02% | 0.17% | -0.09% | 98% | 96% |
CE11.1.6 | 0.13% | -1.32% | -1.58% | 100% | 102% | -0.03% | -1.54% | -1.88% | 100% | 102% |
CE11.1.7 | 0.12% | -1.38% | -1.48% | 102% | 103% | -0.02% | -1.49% | -1.86% | 101% | 102% |
CE11.1.8 | 0.08% | -1.15% | -1.42% | 99% | 101% | -0.02% | -1.21% | -1.62% | 101% | 100% |
SNR based results with ALF off
AI | RA | |||||||||
Test | Y | U | V | EncT | DecT | Y | U | V | EncT | DecT |
CE11.1.1 | 0.04% | -0.03% | -0.01% | 100% | 101% | 0.36% | 0.09% | 0.07% | 101% | 102% |
CE11.1.2 | -0.01% | -0.72% | -0.63% | 100% | 102% | -0.09% | -0.45% | -0.36% | 100% | 102% |
CE11.1.3 | 0.00% | 0.00% | 0.00% | 100% | 102% | -0.01% | 0.00% | 0.00% | 100% | 102% |
CE11.1.4 | 0.01% | 0.00% | 0.00% | 100% | 102% | -0.03% | -0.08% | -0.03% | 102% | 105% |
CE11.1.5 | -0.01% | 0.00% | 0.00% | 99% | 100% | 0.00% | -0.03% | 0.03% | 98% | 99% |
CE11.1.6 | -0.03% | -0.71% | -0.61% | 92%* | 101% | -0.02% | -0.88% | -0.89% | 84%* | 103% |
CE11.1.7 | -0.03% | -0.71% | -0.62% | 108% | 99% | -0.04% | -0.86% | -0.86% | 100% | 102% |
CE11.1.8 | 0.05% | -1.13% | -1.49% | 104% | 103% | -0.02% | -1.64% | -1.65% | 102% | 103% |
LD-B | LD-P | |||||||||
Test | Y | U | V | EncT | DecT | Y | U | V | EncT | DecT |
CE11.1.1 | 0.42% | 0.21% | 0.09% | 101% | 102% | 0.42% | 0.13% | 0.13% | 101% | 101% |
CE11.1.2 | -0.01% | 0.17% | -0.41% | 100% | 102% | -0.09% | -0.43% | -0.61% | 100% | 102% |
CE11.1.3 | -0.04% | 0.01% | -0.22% | 100% | 102% | 0.00% | -0.04% | 0.02% | 100% | 103% |
CE11.1.4 | 0.07% | 0.19% | 0.04% | 100% | 105% | 0.01% | 0.09% | 0.02% | 99% | 103% |
CE11.1.5 | 0.09% | 0.13% | -0.17% | 98% | 98% | -0.06% | 0.10% | -0.14% | 98% | 99% |
CE11.1.6 | 0.04% | -1.12% | -1.74% | 89%* | 103% | -0.02% | -1.77% | -1.94% | 81%* | 102% |
CE11.1.7 | 0.04% | -1.32% | -1.60% | 105% | 105% | -0.06% | -1.85% | -2.01% | 95% | 104% |
CE11.1.8 | -0.03% | -0.58% | -0.50% | 104% | 105% | -0.03% | -0.77% | -0.76% | 103% | 103% |
* Unreliable encoding times.
CE11.2
Test | Proponent(s) | Cross-checker(s) |
CE11.2.1 | Kenneth Andersson kenneth.r.andersson@ericsson.com Anand Meher Kotra Anand.meher.kotra@huawei.com Chia-Ming Tsai | Hyeongmun Jang |
CE11.2.2 | Hyeongmun Jang | Kenneth Andersson kenneth.r.andersson@ericsson.com |
Complexity for luma
Tests | Samples from block bound. modified | Samples from block bound. for deblocking decision | Max num. oper for filtering per line (add/mult/compar/shift) | Max number of oper. for decision per line (add/mult/compar/shift) | Num. line buffers | Worst case complexity increased (Y/N) |
CE11.2.1 | VTM (additional boundary 1+1 but then 1+1 instead of 3+3 for same boundary as VTM) | VTM (additional boundary 3+3 but then 3+3 instead of 4+4 for same boundary as VTM) | VTM (additional boundary 14 (6/2/5/1) but then 14 instead of 56 for same boundary as VTM) | VTM (additional boundary 20/4 (11/0/5/4) /4 per line but then 20/4 instead of 94/8 for same boundary as VTM) | VTM | N |
CE11.2.2 | VTM(no filtering on block boundary of 4xN for EDGE_VER, Nx4 for EDGE_HOR) | VTM | VTM | VTM | VTM | N |
Complexity for chroma
Tests
| Samples from block bound. modified | Samples from block bound. for deblocking decision | Max num. oper for filtering per line (add/mult/compar/shift) | Max number of oper. for decision for 8-sample boundary (add/mult/compar/shift) | Num. line buffers | Worst case complexity increased (Y/N) |
CE11.2.1 | VTM | VTM | VTM | VTM | VTM | N |
CE11.2.2 | VTM (no filtering on block boundary of 2xN for EDGE_VER and Nx2 for EDGE_HOR) | VTM | VTM | VTM | VTM | N |
It is suggested to translate the maximum number of operations per line into the maximum number of operations per sample. This is included in JVET-M0031v4.
VTM3 operates deblocking on an 8x8 grid. If a CU boundary is not on an 8x8 grid, it is not deblocked. If a CU is on an 8x8 grid, furthermore subblocks within that CU are deblocked (provided they are on the 8x8 grid as well). 4xN blocks are deblocked whenever they are coinciding with the 8x8 grid
11.2.1 deblocks on a 4x4 grid, where a boundary is deblocked with VTM filter when the next block or subblock is more than four samples apart, and with a weak and short filter if it is only samples apart.
11.2.2 uses an 8x8 grid and VTM deblocking filter but disables deblocking whenever the boundary would be only four samples apart.
SNR based results (ALF on)
AI | RA | |||||||||
Y | U | V | EncT | DecT | Y | U | V | EncT | DecT | |
CE11.2.1 | 0.02% | 0.00% | 0.00% | 99% | 99% | -0.12% | 0.07% | 0.03% | 98% | 98% |
CE11.2.2 | -0.09% | 0.22% | 0.15% | 100% | 99% | 0.01% | 0.15% | 0.05% | 100% | 99% |
LB | LP | |||||||||
Y | U | V | EncT | DecT | Y | U | V | EncT | DecT | |
CE11.2.1 | -0.14% | -0.17% | -0.01% | 98% | 98% | -0.18% | -0.06% | -0.20% | 98% | 98% |
CE11.2.2 | 0.00% | 0.00% | 0.06% | 100% | 99% | 0.00% | -0.02% | -0.33% | 100% | 100% |
SNR based results (ALF off)
AI | RA | |||||||||
Y | U | V | EncT | DecT | Y | U | V | EncT | DecT | |
CE11.2.1 | -0.05% | -0.01% | 0.00% | 99% | 99% | -0.23% | -0.02% | -0.04% | 98% | 98% |
CE11.2.2 | -0.03% | -0.22% | -0.41% | 100% | 100% | 0.08% | 0.18% | 0.15% | 100% | 100% |
LB | LP | |||||||||
Y | U | V | EncT | DecT | Y | U | V | EncT | DecT | |
CE11.2.1 | -0.21% | 0.04% | -0.23% | 98% | 98% | -0.36% | 0.14% | -0.16% | 97% | 98% |
CE11.2.2 | 0.01% | 0.01% | -0.33% | 100% | 100% | 0.09% | -0.11% | -0.15% | 100% | 100% |
Subjective testing to be done. It is suggested to use an ”AB” comparison where each proposal is compared against VTM3 at same QP.
From the test plan, it would be 96 test cases (2 QPs 34,39, 5 sequences, 8 proposals for 11.1, 2 QPs 30,34, 4 sequences, 2 proposals for 11.2). It would be highly desirable to test both ALF on and ALF off cases. If only one of these cases is possible out of logistic reasons, the ALF off case should be used (as planned in the original CE description). The ALF on bitstreams for subjective testing still need crosscheck.