JVET-J0009 JVET AHG report: Neural Networks in Video Coding (AHG9) [S. Liu, L. Wang, P. Wu, H. Yang]

(Discussion of this report was chaired by GJS and JRO.)

The AHG used the main JVET reflector, jvet@lists.rwth-aachen.de, with [AHG9] in message headers. From February to March, about a dozen emails were exchanged for questions, discussions and expressions of interests. The scope of the AHG was again clarified. The proponent of JVET-I0022 “Convolutional Neural Network Filter for intra frame” made the software available at : https://jvet.hhi.fraunhofer.de/svn/svn_HMJEMSoftware/branches/candidates/HM-16.6-JEM-7.1-AHG9-I0022.

AHG related contributions included:

• JVET-J0034 “AHG9: CNN-based driving of block partitioning for intra slices encoding”, F. Galpin, F. Racapé, P. Bordes, F. Le Léannec, E. François (Technicolor).
• JVET-J0043 “AHG9: Convolutional Neural Network Filter for inter frame”, J. Yao, X. Song, S. Fang, L. Wang (Hikvision).
• JVET-J0076 “AHG9: Results of CNN filter in JVET-I0022 as in-loop filter and post-processing filter”, L. Zhao, X. Li, S. Liu (Tencent).
• JVET-J0037 “Intra Prediction Modes based on Neural Networks”, J. Pfaff, P Helle, D. Maniry, S. Kaltenstadler, B. Stallenberger, P. Merkle, M. Siekmann, H. Schwarz, D. Marpe, T. Wiegand (HHI).

Relevant CfP responses included:

• JVET-J0014 “Description of SDR, HDR and 360° video coding technology proposal by Fraunhofer HHI”, M. Albrecht, C. Bartnik, S. Bosse, J. Brandenburg, B. Bross, J. Erfurt, V. George, P. Haase, P. Helle, C. Helmrich, A. Henkel, T. Hinz, S. de Luxan Hernandez, S. Kaltenstadler, P. Keydel, H. Kirchhoffer, C. Lehmann, W.-Q. Lim, J. Ma, D. Maniry, D. Marpe, P. Merkle, T. Nguyen, J. Pfaff, J. Rasch, R. Rischke, C. Rudat, M. Schaefer, T. Schierl, H. Schwarz, M. Siekmann, R. Skupin, B. Stallenberger, J. Stegemann, K. Suehring, G. Tech, G. Venugopal, S. Walter, A. Wieckowski, T. Wiegand, M. Winken (Fraunhofer HHI).
• JVET-J0016 “Description of SDR video coding technology proposal by KDDI”, K. Kawamura, Y. Kidani, S. Naito (KDDI Corp.)
• JVET-J0018 “Description of SDR video coding technology proposal by MediaTek”, C.-W. Hsu, C.-Y. Chen, T.-D. Chuang, H. Huang, S.-T. Hsiang, C.-C. Chen, M.-S. Chiang, C.-Y. Lai, C.-M. Tsai, Y.-C. Su, Z.-Y. Lin, Y.-L. Hsiao, J. Klopp, I.-H. Wang, Y.-W. Huang, S.-M. Lei (MediaTek).
• JVET-J0022, “Description of SDR, HDR and 360° video coding technology proposal by Qualcomm and Technicolor – medium complexity version”, P. Bordes, Y. Chen, C. Chevance, E. François, F. Galpin, M. Kerdranvat, F. Hiron, P. de Lagrange, F. Le Léannec, K. Naser, T. Poirier, F. Racapé, G. Rath, A. Robert, F. Urban, T. Viellard (Technicolor), Y. Chen, W.-J. Chien, H.-C. Chuang, M. Coban, J. Dong, H. E. Egilmez, N. Hu, M. Karczewicz, A. Ramasubramonian, D. Rusanovskyy, A. Said, V. Seregin, G. Van Der Auwera, K. Zhang, L. Zhang (Qualcomm).
• JVET-J0031 “Description of SDR video coding technology proposal by University of Bristol”, D. Bull, F. Zhang, M. Afonso (Univ. of Bristol).
• JVET-J0032 “Description of SDR video coding technology proposal by University of Science and Technology of China, Peking University, Harbin Institute of Technology, and Wuhan University (IEEE 1857.10 Study Group)”, F. Wu, D. Liu, J. Xu, B. Li, H. Li, Z. Chen, L. Li, F. Chen, Y. Dai, L. Guo, Y. Li, Y. Li, J. Lin, C. Ma, N. Yan (USTC), W. Gao, S. Ma, R. Xiong, Y. Xu, J. Li (Peking Univ.), X. Fan, N. Zhang, Y. Wang, T. Zhang, M. Gao (Harbin Inst. Tech.), Z. Chen, Y. Zhou, X. Pan, Y. Li, F. Liu, Y. Wang (Wuhan Univ.)

The AHG recommended:

• To review all related contributions
• To continue discussions about methodologies and measurements for evaluating neural network coding tools

Aspects of discussion during the presentation included complexity and training processes (questioning whether results are depending on training, usage of original or compressed images, etc.).