JVET-F0032 EE3: Enhanced FRUC Template Matching Mode [Y. Lin, X. Chen, J. An, J. Zheng (HiSilicon)]

This contribution reports test results of JVET-E0035. The following two aspects were tested: 1) bi-directional template matching; 2) selection between bi-prediction and uni-prediction. EE Test1 is designed to test combination of the two aspects while EE Test2 is designed to test aspect 2 only. It is reported that EE Test1 brings 0.36% and 0.14% bit rate savings for RA and LDB configurations, respectively. EE Test2 brings 0.32% and 0.08% bit rate savings for RA and LDB configurations. Both EE tests have almost the same encoding and decoding time as JEM5.0.1.

A modification to the FRUC template matching mode is proposed. An identified issue is that motion information derived by the existing uni-directional template matching is used for bi-prediction in the FRUC template matching mode (allegedly unreasonably). This issue can be solved by the following proposed methods: 1) bi-directional template matching which jointly uses list 0 and list 1 reference pictures; 2) selection between uni-prediction and bi-prediction based on template matching distortion.

get template T₀ matched to current template Tc from list0 reference pictures ( motion info MV0, cost0)

get template T₁ matched to updated template T^’_C from list1 reference pictures (motion info MV1, cost1)

Bi-prediction based on MV0 and MV1

Update current template:

T^’_C = 2*T_C – T₀

cost0 < 0.5*cost1

Uni-prediction based on MV0

get template T₀ (matched to current template) Tc from list0 reference pictures (motion info MV)

get template T₁ matched to current template Tc from list1 reference pictures (motion info MV1)

Bi-prediction based on MV0 and MV1

Proposed FRUC template matching mode

Existing FRUC template matching mode

Y

N

Aspect 1

Aspect 2

One difference compared to current FRUC is the fact that the template based search cannot be implemented in parallel and must be performed sequentially for the two reference pictures.

Note: The Figure above relates to both Aspect 1&2 (Test 1). This requires sequential operation in determining the starting position of the second search.

Test 2 only uses Aspect 2, and determines templates T0 and T1 independently from List 0 and List 1 as in current FRUC, but then determines to use biprediction or uniprediction from one of the lists, depending on the cost criterion:

• Select the bi-, uni-L0-, or uni-L1-prediction according to the min (1.1xcost0, 1.1xcost1, costBi)

Note: Current FRUC always uses biprediction.

Questions recommended to be answered during EE tests.

[Q]: Check gain on top of JEM5.0 (is it additive to JVET-E meeting adoptions)?

[A]: original contribution tested vs JEM4.0 provided 0.38% (RA) / 0.12%(LDB) gain.

On top of JEM5.0, the gain is 0.36% (RA) and 0.14% (LDB) (aspects 1 and 2).

On top of JEM5.0, the gain is 0.32% (RA) and 0.08% (LDB) (only aspect 2).

Summary: Major gain come from selection between uni-prediction and bi-prediction based template matching: 0.3%(RA) / 0.1% (LD); no encoder run-time increment and 1–2% increment for decoder run-time.

Decision: Adopt F0032 Aspect 2. However, the multiplication by factor 1.1 should be implemented in fixed point precision.

JVET-F0032 EE3: Enhanced FRUC Template Matching Mode [Y. Lin, X. Chen, J. An, J. Zheng (HiSilicon)]

Adopt F0032 Aspect 2. However, the multiplication by factor 1.1 should be implemented in fixed point precision.