JVET-L0256 CE9-related: Complexity reduction and bit-width control for bi-directional optical flow (BIO) [X. Xiu, Y. He, Y. Ye (InterDigital)]

This was discussed Saturday 2105 (GJS).

Bi-directional optical flow (BIO) is include in benchmark set (BMS)-2.1. The current BIO implementation in the BMS-2.1 requires 33-bit multiplier and has the maximal bit-width of 43 bits for intermediate parameters. When BIO is enabled, the number of multiplications is reportedly 329% of the worst-case number of multiplications for regular bi-prediction for 4x4 blocks, representing a significant increase in worst-case computation complexity.

In this contribution, two methods are proposed to address the BIO’s complexity issues.

• A bit-width control method is proposed to ensure BIO can be implemented with at most a 15-bit multiplier and the intermediate values are within the 32-bit range.
• A method is proposed to lower the BIO’s computational complexity by using bilinear filters to interpolate prediction samples and reducing the area of extended region for BIO derivation. Further, BIO is disabled if either of the following is true: 1) the CU has height equal to 4 or the CUs is in size of 4×8, or 2) the CU is coded with sub-block modes.

The worst-case number of multiplications involved in the BIO can reportedly be reduced to 103% of that of regular bi-prediction by the proposed changes.

Experimental results reportedly show that compared to the VTM-2.1, the combination of the proposed changes provides average {Y, U, V} BD-rate savings of {1.28%, 0.52% and 0.37%} with encoding and decoding time of 102% and 102%, respectively. The existing BIO implementation in the BMS-2.1 reportedly provides an average {Y, U, V} BD-rate saving of {1.41%, 0.58%, 0.40%} with encoding and decoding time of 103% and 113%, respectively. Based on the simulation results, it is asserted that with the proposed methods, BIO becomes an implementable tool with a desirable performance vs. complexity trade-off.

Text was not provided, but was being prepared at the time of this discussion.

It was reported that the existing design cannot be implemented readily with SIMD operations on typical processors due to bit width concerns. The results reported for the proposed method used some SIMD operations. There was discussion of whether the SIMD usage was appropriate, and it was said that the other inter prediction processes in the software have used SIMD implementation as well.

Similar approaches to complexity reduction had been suggested in different proposals that had been developed independently.

A participant commented that a number of aspects of this are new and should be studied.

It was also commented that there may be overlap in gain between BIO and DMVR. Others said that the gains do not substantially interfere with each other.

BIO has been in the BMS previously and has been studied in some form for years. This drop the complexity of the proposal very substantially.

Decision: Adopt (subject to text review).

Further study of additional refinements was encouraged.