JVET-T0092 AHG9/AHG11: Neural network based super resolution SEI [T. Chujoh, E. Sasaki, T. Ikai (Sharp)]
This contribution was discussed in JVET session 12 at 0640 on Tuesday 13 October (chaired by GJS & JRO).
In this contribution, a framework for introducing a super-resolution post-filter using NN (Neural Network) is proposed. In VVC, RPR (Reference Picture Re-sampling) has been introduced. As a result of some experiments to change the resolution of the whole sequence, in several 4K sequences, there were some coding gains. In this experiment, the up-sampling filter is used the interpolation filter of VVC. If some super-resolution post-filters using NN are introduced, the visual quality will be more improved. For example, MPEG NNR which is under development in SC29/WG4 can describe neural networks efficiently. In this contribution, an SEI message which can encode and decode a neural network for super-resolution is proposed.
The contributor recommended that AHG studies should consider a method of using RPR.
There was some odd behaviour for chroma.
Visual quality investigation was recommended.
It was commented that there could be some denoising effect, and that it may be desirable to study use of the denoising prefilter in VTM.
It was further commented that there has been a bug in the temporal prefilter software’s interaction with another feature, which will soon be fixed.
It was commented that there was a University of Bristol contribution to the CfP, and perhaps another contribution at the CfP stage, that contained some similarities.
The “convex hull” style of comparison across resolutions was suggested to be considered.
See also JVET-T0096.
JVET-T0092 AHG9/AHG11: Neural network based super resolution SEI [T. Chujoh, E. Sasaki, T. Ikai (Sharp)]
In this contribution, a framework for introducing a super-resolution post-filter using NN (neural network) is proposed. In VVC, RPR (reference picture resampling) has been introduced. As a result of some experiments to change the resolution of the whole sequence, in several 4K sequences, there were some coding gains. In this experiment, the up-sampling filter is used the interpolation filter of VVC. If some super-resolution post-filters using NN are introduced, the visual quality will be more improved. For example, MPEG NNR which is discussed in SC29/WG4 can describe neural networks efficiently. In this contribution, a simple SEI which can encode and decode a neural network for super-resolution is proposed. The contribution recommended that AHG11 should consider a method of using RPR.
See also notes for this contribution in section 5.2 and also for JVET-T0096.
See section 7.1 for notes of a joint meeting discussion. Further detailed presentation was not deemed necessary at this meeting.