JVET-V0004 JVET AHG report: Test material and visual assessment (AHG4) [V. Baroncini, T. Suzuki, M. Wien, E. François, S. Liu, A. Norkin, A. Segall, P. Topiwala, S. Wenger, Y. Ye]

AHG meetings were held related to the preparation of the VVC verification tests in the SDR HD, 360° video and HDR categories on 2021-02-18 and 2021-03-25, respectively. The reports of these meetings are provided in the input documents JVET-V0042 and JVET-V0044.

The status of the discussion in the SDR HD and 360° video categories was reflected in the verification test plan document JVET-U2021. In the SDR HD low delay category, a set of 6 sequences was selected including three test sequences representing the conversational application scenario and three test sequences representing the gaming-type application scenario. Out of the four test sequences under consideration in the conversational application scenario, the verification test coordinators selected three as per request of the AHG meeting. The resulting set considered for subjective assessment includes the test sequence Beatriz, OfficeWalkAtWall, and OfficeWalkCeiling. For the SDR HD and 360° video categories, bitstreams for the rate points and configurations defined in JVET-U2021 were generated and verified for HM-16.22 and VTM-11.0 by JVET members in an off-line activity. For the SDR RA category, additional bitstreams for VVenC-0.3 bitstreams were provided as an additional test case.

The verification test coordinators are in contact with laboratories who offered support in contributing to the testing procedure. However, due to the COVID-19 situation, the activities for formal subjective assessments have been strongly reduced in many countries which impacts the progress of the VVC verification tests. A recent update on the local conditions in Italy opened the possibility for performing visual tests which is currently ongoing.

In the HDR category, efforts have been extended to identify suitable test sequences for both, PQ and HLG, including experts viewing sessions for determination of a QP range suitable for covering the intended subjective quality range.

Test sequences

The test sequences used for CfP/CTC are available on ftp://jvet@ftp.ient.rwth-aachen.de in directory “/jvet-cfp” (accredited members of JVET may contact the JVET chairs for login information).

Due to copyright restrictions, the JVET database of test sequences is only available to accredited members of JVET (i.e. members of ISO/IEC MPEG and ITU-T VCEG).

Copyright licenses have been updated for the following HDR PQ content located at ftp://jvet@ftp.ient.rwth-aachen.de, in folder /ctc/hdr:

• Market3_1920x1080p_50_10b_pq_709_ct2020_420_rev1.zip
• SunRise_1920x1080p_25_10b_pq_709_ct2020_420_rev1.7z

The following related contributions were submitted.

• JVET-V0041 “AHG4: Status Report on SDR HD and 360 Video Verification Test Preparation” [M. Wien, V. Baroncini, Y. Ye (AHG coordinators)]
• JVET-V0042 “AHG4: Agenda and report of the AHG meeting on the SDR HD and 360 video verification test preparation on 2021-02-18” [M. Wien, V. Baroncini, Y. Ye (AHG coordinators)]
• JVET-V0043 “ AHG4: Status Report on HDR Video Verification Test Preparation” [A. Segall, M. Wien, V. Baroncini, K. Andersson]
• JVET-V0044 “AHG4: Agenda and report of the AHG meeting on the HDR verification test preparation on 2021-03-25” [A. Segall, M. Wien, V. Baroncini (AHG coordinators)]
• JVET-V0045 “Aerial 4K HDR (HLG) Test Data” [Pankaj Topiwala (FastVDO)]

The AHG reports JVET-V0042 and JVET-V0044 include a summary of the discussion of JVET-V0041, JVET-V0043. The contributions related to the verification tests should be further discussed during the meeting.

The AHG recommended:

• To review the input contributions related to the verification test preparation.
• To continue to discuss and to update the non-finalized categories of the verification test plan.
• To create a BoG to progress the update of the verification test plan.
• To collect volunteers to conduct the verification test, including volunteers to encode.
• To review the set of available test sequences for the verification tests and potentially collect more test sequences with a variety of content.
• To continue to collect new test sequences available for JVET with licensing statement.

Some expert viewing was suggested to be held during the meeting.

It was further suggested to merge the available test data into one ftp site (including previous JCT-VC test sets).