JVET-AC0310 EE1-related: Combination of EE1-1.5 and EE1-1.7 [Y. Li, K. Zhang, L. Zhang (Bytedance), D. Liu, J. Ström, M. Damghanian, P. Wennersten, K. Andersson (Ericsson)] [late]

This contribution presents the results of a combination test of EE1-1.5.3 in JVET-AC0089 and EE1-1.7.1 in JVET-AC0177. EE1-1.5.3 proposes to reduce model memory by using a shared model for the in-loop filtering of different types of slices. EE1-1.7.1 proposes to improve the in-loop filtering performance by introducing an additional CNN model. The combined test is to achieve reduced model memory and improved performance compared with NNVC-3.0 filter set #1.

BD-rate changes of {Y, Cb, Cr} of the combined test anchored on EE1-1.5.3 is {-0.73%, 0.06%, 0.06%} for RA.

BD-rate changes of {Y, Cb, Cr} of the combined test anchored on NNVC-3.0 filter set #1 is {-0.86%, -0.18%, -1.35%} for RA.

BD-rate changes of {Y, Cb, Cr} of the combined test anchored on NNVC-3.0 anchor is {-10.26%, -20.88%, -21.43%} for RA.

BD-rate changes of {Y, Cb, Cr} of EE1-1.7.1 anchored on NNVC-3.0 filter set #1 is {-0.67%, 0.03%, 0.05%} for RA.

From results reported above, it is asserted that the method proposed in EE1-1.7.1 can bring similar gains on top of NNVC-3.0 filter set #1 and EE1-1.5.3.

The combination is made by applying 1.7.1 to the three highest temporal layers, 1.5.3. to the remaining layers. This means that different models are used for different temporal layers. In total, 3 models (1 for the higher layers, 2 for the lower ones)

It was commented that increasing the PSNR quality of higher temporal might not be as much give subjective improvement.

It was commented by different experts that the introduction of an additional model is undesirable.

E. Alshina commented that the interesting aspect of 1.7.1 is the introduction of filtering taking additional input from reference frames. To allow experts investigating that technology, it was suggested to include in NNVC software (in the version of combination with 1.5.3), but turn it off by default for comparison with other proposals not using such an approach in the EE.

Decision: Adopt JVET-AC0310, but make 1.5.3 the default configuration, and allow disabling 1.7.1 component. Further, the method shall be changed to use the actual reference frame (not the unfiltered decoded frame).