JVET-T0045 AHG4: Agenda and report of the AHG meeting on the SDR HD Verification Tests on 2020-09-03 [M. Wien, V. Baroncini]

This was a report of an interim AHG meeting. The meeting was held online on Thursday, 2020-09-03, at 1600 UTC.

It was noted that the gains observed for SDR HD in the low-delay (LD) configuration appear to be lower than what has been reported for the SDR UHD sequences in the random-access (RA) configuration before.

It was commented that this may potentially be related to the LD configuration that has been used in this setup. It was suggested to investigate this issue by looking at the performance of the same sequences using the RA configuration and by downscaling and coding UHD sequences using LD which have been assessed under RA configuration before.

It was suggested to do testing of SDR HD content in both, LD and RA configurations. It was asserted that RA coding of HD content is a relevant application case which should be addressed in the VVC verification tests.

In terms of test content, it was suggested broaden the set of test sequences under consideration and to seek for suitable content representing conversational applications, as well as mobile streaming applications.

In the AHG meeting it was commented that the QP settings used in the LD-B configuration from the JVET common test conditions might not be optimized for visual assessment. It was noted that for HD LD, the behaviour of the MOS curves is following the behaviour of the PSNR curves much closer than for the UHD RA case where the visual quality was found to be higher, by tendency, compared to the PSNR curve. It was suggested to investigate the LD configuration files in order to address potential issues with visual quality. It was mentioned that the GOP size used in the LD configuration could impact the visual quality. A GOP size change from 4 to 8 pictures in the VTM was adopted into the common testing conditions based on contribution JVET-P0345. The LD configuration of HM-16.20 uses a GOP size of 4. It was suggested that HM and VTM should be comparable in terms of configuration (if both specifications support the corresponding settings).

It was commented in the AHG meeting that the content currently under investigation does not match the LD application case well (the Witcher1 test sequence might be closest). It was noted that the LD configuration should be demonstrated on conversational and gaming content.

Next steps identified as needed were discussed as follows:

• Investigate the appropriateness of the available LD configuration for subjective evaluation, e.g. by expert viewing.
• See contribution JVET-T0097 on RA configuration.
• Include SDR HD RA as a verification test category, besides SDR HD LD. The test sequences for both cases could be selected independently. The current test sequences for HD VT are not typical for low delay applications.
  • More conferencing content is needed. See JVET-T0060, which proposes new test sequences. Another test sequence had recently been provided by Ericsson, and it was requested for a corresponding document to be provided, which was later uploaded as JVET-T0120.
  • A set of tests for HD RA is desired.
• It was commented that HM 16.22 should be used to be able to use the latest configurations of LD and RA GOP structures.
• Seek test sequences suitable for visual assessment both types of application scenarios, conversational/gaming (LD) and streaming (RA). Some sequences of the current SDR HD test set might be suitable if encoded with the RA configuration. JVET experts are invited to suggest and provide test sequences to the verification test coordinators.

It was planned to conduct remote expert viewing during the meeting to identify the suitable test points in the HD SDR RA category.

It was reported that the verification test on HD SDR RA could likely be conducted during the upcoming meeting cycle (if the pandemic situation allows).