JVET-U0119 Dry run subjective assessment of SDR HD and 360° video verification tests [V. Baroncini, M. Wien]

This document provides a report on results of subjective quality evaluation for test sequences under consideration for the VVC verification test. The QP selection for the test sequences under consideration in the SDR HD and 360° video categories as proposed in JVET-U0041 is tested for suitability for use in the VVC verification test according to the verification test plan JVET-T2009. Furthermore, candidate QPs for gaming-type test sequences are proposed and evaluated. In the SDR HD category, subset of the candidate test sequences and candidate rate points for the low delay and random access configurations are evaluated. In the 360° video category, mp4 files of viewports of the test sequences encoded using the PERP projection format are assessed. Here, the full set of rate points was included in the tests. The results indicate that the candidate QPs provide a coverage of the quality range from quite bad to very good quality and some adjustments of the QPs are suggested. In the SDR HD category, overall rate saving estimates cannot be provided from the data. Only potential isolated rate savings are estimated for rate points with overlapping confidence intervals. In the 360° video category, estimated rate savings between approximately 30% and 60% are reported.

GOP size 32 is being used for VVC RA.

It was noted that it was an open issue whether the GOP size for LD should be 4 or 8. It had also been noted in JVET-U0041 that, based on visual inspection of the verification test coordinators, a GOP size of 4 pictures was suggested to be used in this test category for both HEVC and VVC. JVET-U0041 reported on bitstreams for SDR HD conversational content that were provided using the GOP4 configuration, while in JVET-T2009 a set of seven SDR HD test sequences for gaming-type applications were described using the GOP8 configuration.

JVET-U0044 said that it was emphasized that for the low delay configuration, a GOP size of 4 was found to be visually better than GOP8. This assessment was done on the set of sequences previously considered for the SDR HD LD setting. It was pointed out that the bitstreams used for visual assessment before that AHG meeting had been encoded using GOP8. The previously established CTC conditions have specified the GOP8 configuration, and the GOP8 configuration may have better PSNR performance. JVET-U0044 said there was an action to be taken to perform encodings with the GOP4 configuration for VTM-11.0 and HM-16.22 and conduct a visual assessment for confirmation.

Post-meeting note: This GOP4 vs. GOP8 question was apparently not resolved at the meeting, and it might thus need to be left to the discretion of the test coordinators or further discussed in AHG work in preparation for the verification tests. Simply making sure the HM and VTM are both configured the same way in this regard is likely to be more important for comparison purposes than which of the two GOP sizes is used.

MCTF was not planned to be used for LD and 360° since not confident about the effects in these cases.

It was reported that it may be possible to perform the remaining planned tests within the period prior to the next meeting.