JVET-W0081 AHG11: BD-rate gains vs complexity of NN-based intra prediction [T. Dumas, F. Galpin, P. Bordes, F. Le Léannec (InterDigital)]
This contribution presents a low complexity version of the neural network-based intra prediction proposed in last meeting.
Mean BD-rate reductions of -2.88% -2.33% -2.42% and -1.44% -0.52% -0.97% in AI and RA configurations respectively are reported when comparing VTM-11.0 including the low complexity version of the neural network-based intra prediction mode w.r.t VTM-11.0 (not VTM-11.0-NNVC). The complexity (in MAC/pixel) of the low complexity version of the neural network-based intra prediction mode is about 12 times smaller than its regular version.
Low complexity is via sparse training of weights (enforcing zero weights).
8 different networks are used for different block sizes.
Question: For which block sizes most efficient? Medium size, such as 8x8
Could this approach of complexity reduction also be applied to CNN? Likely would not work, as properties of filters, and their effect on dedicated spatial positions might get lost.
How is the training for sparsity implemented? How to deal with the fact that the non-zero weights could be randomly positioned in the network? Try to start at certain positions, and increase threshold gradually.
They also reportedly investigated a version with 16 bit integer, with almost no difference in results.
It was agreed to investigate this in an EE.