Back to Search Document details
32nd Meeting: Hannover, DE, October 2023 2023-10-17 13:11
EE1-1.2.2: Content-adaptive LOP filter
Authors: Ruiying Yang Nokia Maria Santamaria Nokia Francesco Cricri Nokia Honglei Zhang Nokia Jani Lainema Nokia Miska M. Hannuksela Nokia Antti Hallapuro Nokia
Abstract
This contribution reports the results for the EE1-1.2.2 test which studies an integrated signalling mechanism for the content-adaptive neural network loop-filter in JVET-AE0093, and uses as basis the LOP1 loop-filter. It is reported that a new type of adaptation parameter set (APS), called neural network filter update APS, was designed and implemented. In JVET-AE0093, the inference was conducted on top of NNVC 5.0, using SADL library with half rounding enabled. The attained coding gains were -1.21% (Y), -6.43% (Cb) and -5.52% (Cr). Nevertheless, it was observed that the deactivation of half rounding in SADL library led to a reduction in coding gains by approximately 1%. Therefore, this contribution also studies data-driven quantization as means to improve the inference performance with SADL library and int16 precision. It is reported that, the average coding gains of this approach are -1.19% (Y), -6.40% (Cb) and -5.49% (Cr) with respect to the NNVC 5.0 anchor (NN intra tool ON and LOP1 tool ON). Furthermore, a simulation of signalling the update for each intra frame was performed and it is reported that the average coding gains are -0.34% (Y), -5.52% (Cb) and -4.75% (Cr) with respect to the NNVC 5.0 anchor.
Meeting Notes
JVET-AF0056 EE1-1.2.2: Content-adaptive LOP filter [R. Yang, M. Santamaria, F. Cricri, H. Zhang, J. Lainema, M. M. Hannuksela, A. Hallapuro (Nokia)]

This contribution reports the results for the EE1-1.2.2 test which studies an integrated signalling mechanism for the content-adaptive neural network loop-filter in JVET-AE0093, and uses as basis the LOP1 loop-filter. It is reported that a new type of adaptation parameter set (APS), called neural network filter update APS, was designed and implemented. In JVET-AE0093, the inference was conducted on top of NNVC 5.0, using SADL library with half rounding enabled. The attained coding gains were -1.21% (Y), -6.43% (Cb) and -5.52% (Cr). Nevertheless, it was observed that the deactivation of half rounding in SADL library led to a reduction in coding gains by approximately 1%. Therefore, this contribution also studies data-driven quantization as means to improve the inference performance with SADL library and int16 precision. It is reported that, the average coding gains of this approach are -1.19% (Y), -6.40% (Cb) and -5.49% (Cr) with respect to the NNVC 5.0 anchor (NN intra tool ON and LOP1 tool ON). Furthermore, a simulation of signalling the update for each intra frame was performed and it is reported that the average coding gains are -0.34% (Y), -5.52% (Cb) and -4.75% (Cr) with respect to the NNVC 5.0 anchor.

From EE report: Continue EE, see under JVET-AF0023

Was further presented Tue 17 Oct. at 1330 (chaired by JRO)

In the new version, signalling was enabled for each RA points (I picture), which caused a drop of gain down to 0.3% due to additional bit overhead. However, the same filter parameters were signalled for each I period, trained for the entire sequence. The more realistic scenario would be training for each I period separately, which would probably improve the performance.

It was agreed to investigate this in an EE.

New LOP should be used, training crosscheck only for chunks, not the base model.

Cited:
JVET-AF0119 X JVET-AF0216 X
ARCHIVE :
JVET-AF0056.pptx JVET-AF0056_v1.docx JVET-AF0056_v2.docx VTM_NNVC5.0_vs_Overfitting_main_results.xlsm VTM_NNVC5.0_vs_Overfitting_signal_wu_for_each_ra.xlsm
Decisions
New LOP should be used, training crosscheck only for chunks, not the base model.
Citation