Back to Search Document details
29th Meeting: by teleconference, DE, January 2023 2023-01-12 23:57
EE1-1.10: Complexity Reduction on Neural-Network Loop Filter
Authors: Jay N. Shingala Ittiam Ajay Shyam Ittiam A. Suneia Ittiam S. P. Badya Ittiam Tong Shao Dolby A. Arora Dolby P. Yin Dolby Sean McCarthy Dolby
Abstract
This contribution reports the experimental results of EE1-1.10. The models for complexity reduction in this EE test are originally proposed in JVET-AA0080 and JVET-AB0136, with the introduction of parallel fusion of deblocked samples and NNLF outputs for further improvement. In EE1-1.10.1, the model with CP decomposition plus fusing adjacent 1x1 convolution is tested. In EE1-1.10.2, the model with CP decomposition, fusing adjacent 1x1 convolution and split architecture for luma and chroma components is tested. The models are implemented on top of NNVC common software NNVC-3.0. They are also implemented with fixed point using SADL. BD-Rate results are reported compared to NNVC-3.0 anchor. While the JVET-X0140 low complexity model achieves about 5.16% luma gain for AI and 4.98% for RA with worse case block level complexity of 33.6 KMAC/Pixel, the proposed models can achieve better trade-off with lower complexity. The AI and RA results of {Y, Cb, Cr} BD-Rate gain are as follows:
Meeting Notes
JVET-AC0106 EE1-1.10: Complexity Reduction on Neural-Network Loop Filter [J. N. Shingala, A. Shyam, A. Suneia, S. P. Badya (Ittiam), T. Shao, A. Arora, P. Yin, S. McCarthy (Dolby)]

This contribution reports the experimental results of EE1-1.10. The models for complexity reduction in this EE test are originally proposed in JVET-AA0080 and JVET-AB0136, with the introduction of parallel fusion of deblocked samples and NNLF outputs for further improvement. In EE1-1.10.1, the model with CP decomposition plus fusing adjacent 1x1 convolution is tested. In EE1-1.10.2, the model with CP decomposition, fusing adjacent 1x1 convolution and split architecture for luma and chroma components is tested. The models are implemented on top of NNVC common software NNVC-3.0. They are also implemented with fixed point using SADL. BD-Rate results are reported compared to NNVC-3.0 anchor. While the JVET-X0140 low complexity model achieves about 5.16% luma gain for AI and 4.98% for RA with worse case block level complexity of 33.6 KMAC/Pixel, the proposed models can achieve better trade-off with lower complexity. The AI and RA results of {Y, Cb, Cr} BD-Rate gain are as follows:

EE1-1.10.1: CP-decomposition with fusing, worst case block level complexity of 16.2 KMAC/Pixel

  • LibTensorflow, fp32: AI {-4.65%, -5.66%, -5.03%}, RA {-5.05%, -6.29%, -4.46%}
  • SADL, fp32: AI {-4.64%, -5.62%, -4.97%}, RA {-5.02%, -6.36%, -4.38%}
  • SADL, int32: AI {-4.62%, -5.62%, -4.96%}, RA {-5.01%, -6.24%, -4.32%}

EE1-1.10.2: CP-decomposition with fusing and split luma chroma (24L, 8C), worst case block level complexity of 17.7 KMAC/Pixel

  • LibTensorflow, fp32: AI {-4.94%, -7.76%, -7.69%}, RA {-5.43%, -7.79%, -6.57%}
  • SADL, fp32: AI {-4.91%, -7.21%, -6.97%}, RA {-5.37%, -7.61%, -6.31%}
  • SADL, int32: AI {-4.88%, -7.24%, -6.99%}, RA {-5.34%, -7.64%, -6.34%}

This was presented in session 12.

The method uses 4 models, each of size about 57K parameters, 17.7 kMAC/Pix

It was agreed to study this in a next EE, including:

  • Training crosscheck
  • Int16 implementation
  • If possible, SIMD implementation in SADL

The possibility of further reduction of number of computations should be considered.

Cited:
JVET-AC0241 X
ARCHIVE :
JVET-AC0106-NNLF-complexity_v1.pdf JVET-AC0106_NNLF_complexity.docx JVET-AC0106_Split24L8C_SADLfp32.xlsm JVET-AC0106_Split24L8C_SADLint32Q14.xlsm JVET-AC0106_Split24L8C_TF.xlsm JVET-AC0106_X0140_SADLfp32.xlsm JVET-AC0106_X0140_SADLint32Q14.xlsm JVET-AC0106_X0140_TF.xlsm
Decisions
The possibility of further reduction of number of computations should be considered.
Citation