JVET-E0060 EE3-JVET-D0046: High precision FRUC with additional candidates [A. Robert, F. Le Léannec, T. Poirier (Technicolor)]

This contribution responds to the Exploratory Experiments 3, part D0046. It tests independently the features of D0046, then reduces the complexity by adjusting the number and the place in the FRUC lists of the added candidates and by removing several candidates from the FRUC lists of sub-blocks of merge CUs. High precision refinement improves only Low Delay P configuration (0.13%), but increases the encoding and decoding runtimes (1% to 3%). Candidate addition provides BD-rate gains (0.23% RA, 0.64% LDB and 0.77% LDP) with increases of encoding (1% RA, 5% LDB and 3% LDP) and decoding (1–2%) runtimes. However, in Test #3, the coding gains are preserved even improved with a complexity reduction of both encoding and decoding runtimes that remain close to 100%.

Overall, four (2 AMVP and 2 spatial) and two spatial additional motion vector candidates are used for AMVP and merge CUs respectively. Two additional spatial candidates are used for each sub-block of a merge CU, but several of initial ones have been removed (zero and unilateral motion vectors, bottom-right co-located candidates and up to 24 ATMVP candidates).

From the EE summary report JVET-E0010:

High precision FRUC with additional candidates: The technology contains:

• Increase of the precision in FRUC until the finer internal one (currently 1/16);
• Addition of Motion Vector Candidates in FRUC, by adding the 2 AMVP candidates in the front of the FRUC candidate list for AMVP CUs, and up to 5 Spatial Candidate(s) in the end of FRUC candidate list, both for entire CUs, and FRUC CUs’ sub-blocks.
• Remove some FRUC candidates from the list

Questions recommended to be answered during EE tests:

[Q]: What is the performance impact from increment the precision in FRUC" up to 1/16?

[A]: Test #1 was designed to answer this question. Performance impact from this change is relatively low: 0.0% (RA) / 0.1% (LD) / −0.1%(LDP).

[Q]: How many "Additional of Motion Vector Candidates" in total?

[A]: Based on test 3, the numbers of added candidates are:

• +4 for AMVP CUs
• Between +2 and −31 for Merge CUs (a negative number meaning that less candidates are used than in JEM-4.0).

[Q]: Can some candidates (ex. spatial) be re-ordered?

[A]: Indeed, for sub-blocks of merge CUs, test 2 re-orders the candidate list as follows, compared to test 1.2:

• top and left neighbouring motion vectors are moved from ending place to 2nd place.
• added top-left and top-right neighbouring motion vectors are moved from ending place to 3rd place.

RA: −0.2% (ET 1.01, DT 1.01) RA: −0.2% (ET 0.97, DT 1.00)

LD: −0.6% (ET 1.05, DT 1.02) LD: −0.7% (ET 1.01, DT 1.01)

LDP: −0.8% (ET 1.03, DT 1.01) LDP: −0.8% (ET 0.99, DT 1.01)

[Q]: Can some candidates (among initial FRUC candidates or added spatial candidates) be removed?

[A]: Test #3 was designed to answer this question:

RA: −0.2% (ET 0.97, DT 1.00) RA: −0.2% (ET 1.00, DT 0.99)

LD: −0.7% (ET 1.01, DT 1.01) LD: −0.6% (ET 1.04, DT 0.98)

LDP: −0.8% (ET 0.99, DT 1.01) LDP: −0.8% (ET 1.03, DT 1.00)

Despite the low complexity decrease obtained with the removals of test 3 (1% encoder and 2% decoder time), the number of candidates for sub-blocks is drastically reduced and smoothed.

Test 3 reduces the theoretical complexity and simplifies the design of the FRUC tool.

Summary: 0.2%(RA) / 0.7% (LD) / 0.8%(LDP) gain is observed with run-time increment −3% to 1% (encoder) and 0–1% (decoder. (test2)

0.2%(RA) / 0.6% (LD) / 0.8%(LDP) gain is observed with run-time increment 0/4% (encoder) and −2%/0% (decoder) (test3)

Discussion in JVET: Configuration from Test 3 is a simplification by reducing the number of candidates, and still gives compression benefit, in particular for the LD cases. This is also confirmed by a crosschecker.

Decision: Adopt JVET-E0060 Test 3 configuration.

It was clarified again by the crosschecker that the test 3 configuration does not include an increase of precision (unlike Test 1.1).

JVET-E0060 EE3-JVET-D0046: High precision FRUC with additional candidates [A. Robert, F. Le Léannec, T. Poirier (Technicolor)]

Adopt JVET-E0060 Test 3 configuration