JVET-T0103 Information on and analysis of the VVC encoders in the SDR UHD verification test [C. Helmrich, B. Bross, J. Pfaff, H. Schwarz, D. Marpe, T. Wiegand (HHI)] [late]
This was discussed in session 16 at 0835 on Wednesday 14 October (chaired by GJS & JRO).
This contribution provides further information on the configuration of the VVC reference encoder (VTM) and the alternative encoder (VVenC) evaluated in the VVC verification test for SDR UHD content, specified in JVET-S2009 (test plan, draft 3). Based on an analysis of the bitstreams and Bjøntegaard delta-rate values obtained using several video quality assessment (VQA) methods, the following conclusions can be drawn:
- JVET-S2009 specifies VTM version 9.0 for the SDR UHD verification test, but the decoded video files resulting from newer VTM 10.0 encoded bitstreams were found to be identical to those resulting from the VTM 9.0 encodings. Thus, the SDR UHD verification test was, effectively, conducted using the VTM 10.0 software release, which was the latest VTM version as of late September 2020.
- The results of the SDR UHD verification test, reported in JVET-T0097, indicate that VVenC 0.1.0 (VTM 10 decoder compliant) tends to outperform the VTM encoder in terms of visual quality. It was found that only some of the VQA methods used for Bjøntegaard delta-rate evaluation – namely, the XPSNR, SSIM, and MS-SSIM methods – succeed in modelling this tendency with satisfactory accuracy. The PSNR and VMAF metrics, in particular, fail to model the subjective judgments well.
The VVenC encodings include optimizations for subjective video quality and encoder-only efficiency improvements, namely,
- Signal and in-GOP frame index dependent perceptual QP adaptation (QPA) based on the published JVET documents JVET-H0047, JVET-K0206, and JVET-M0091, QPA related ALF encoder changes JVET-L0181, and more recent XPSNR related enhancements to the spatiotemporal QPA algorithm, as further described in the contribution.
- Encoder-only deblocking optimization (EDO) accounting for the effect of deblocking on the reconstructed-video signal distortion within the encoding parameter search-loop of each coded frame,
- Encoder-only GOP based motion compensated temporal filtering (MCTF) as a denoising pre-filter to improve the motion prediction performance and thus provide objective and subjective gains.
See further information in the contribution document and also the notes for JVET-T0069 regarding “VMAF” having different models and different versions.
A plug-in for the XPSNR (version 1.0, initial release, as a plug-in for FFmpeg) is available at https://github.com/fraunhoferhhi/xpsnr/tags.
It was commented that there is a mode decision feature in the VTM for partially accounting for deblocking in the mode decision and that this is not enabled in the current CTC, and perhaps it should be (see JVET-M0428, reporting 0.7% for RA). It was mentioned that this is not available in the HM, and was suggested that we should not use it in VTM-to-HM comparisons unless it is available in both HM and VTM, as it could hypothetically apply to both contexts.
It was commented that part of the subjective benefit could be from the MCTF, which was not enabled in the VTM test cases of the verification test. It is further commented that this design of the test was by purpose, as the non-normative configuration of HM and VTM was designed as similar as possible (under the constraints and tools implemented in the respective standard). If it was enabled in VTM, it should also be enabled in HM.
The proponent agreed that some of the visual benefit might be due to MCTF, but also said that in his opinion QP adaptation also provided a substantial benefit in some cases.