JVET-AF0005 JVET AHG report: Conformance testing (AHG5) [I. Moccagatta (chair), F. Bossen, K. Kawamura, P. de Lagrange, T. Ikai, S. Iwamura, H.-J. Jhu, S. Paluri, K. Sühring, Y. Yu (vice chairs)]

The AHG communication was to be conducted through the main JVET reflector, jvet@lists.rwth-aachen.de, with [AHG5] in message headers. However, no correspondence marked as AHG5 was sent between the 31^st and 32^nd meetings.

Timeline

The progress on the Conformance testing specification was proceeding per the timeline below:

• VVCv1 conformance:
  • ISO/IEC FDIS 23090-15 issued from 2021-10 meeting, FDIS registered for formal approval 2022-07-11, FDIS ballot closed 2022-11-04, standard published 2022-11-24
  • H.266.1 V1 Consent 2022-01-28, Last Call began 2022-04-01, Approved 2022-04-29, pre-published 2022-05-17, standard published 2022-07-12.
• VVCv2 conformance:
  • ISO/IEC 23090-15/Amd.1 CDAM: 2021-10
  • ISO/IEC 23090-15/Amd.1 DAM: 2022-01
  • DAM ballot closed 2022-11-15
  • ISO/IEC FDIS 23090-15:202x 2nd edition text output of 2023-01, preparation delayed to 2023-09
  • H.266.1 V2 forwarded by JVET and Q6/16 for ITU-T Consent: 2023-07
  • H.266.1 V2 approved 2023-09-13
  • H.266.2 V2 pre-published 2023-10-06

Status on bitstream submission

The status at the time of preparation of this report was as follows:

• Conformance bitstreams for VVC:
  • 104 bitstream categories had been identified
  • At least one bitstream had been submitted in each identified category
  • 283 total bitstreams had been provided, checked, and made available
  • No changes in bitstream between 31^st and 32^nd meeting.
• Conformance bitstreams for VVC operation range extensions:
  • 57 bitstream categories had been identified
  • 1 bitstream of 1 identified category had been re-generated
  • 128 bitstreams of 57 identified categories had been cross-checked and uploaded.
  • No changes between 31^st and 32^nd meeting.

Activities and Discussion

The AHG activities were on schedule with the preliminary timeline shown above.

VVC activities:

• In the 2^nd edition company names and emails had been removed from the packages, 2^nd edition packages are available at https://www.itu.int/wftp3/av-arch/jvet-site/bitstream_exchange/VVC2ndEd/

VVC operation range extensions activities:

• In the 2^nd edition company names and emails have been removed from the packages, 2^nd edition packages are available at https://www.itu.int/wftp3/av-arch/jvet-site/bitstream_exchange/VVC2ndEd/

VVC Multilayer activities:

• Six of the 7 additional conformance bitstreams for VVC multilayer configurations in JVET-AE2028 decoded correctly using VTM-20.2 are available in https://www.itu.int/wftp3/av-arch/jvet-site/bitstream_exchange/VVCMultilayer/under_test/VTM-20.2/.
• Volunteers to cross check these 6 conformance bitstreams for VVC multilayer configurations have been identified and cross-check is ongoing.
• According to the proponents, the VVC multilayer configurations bitstream in JVET-AE0111 can be decoded by VTM-20.2 and a volunteer to cross-check the bitstream has been identified. This bitstream has been added to JVET-AE2028 and will be uploaded to https://www.itu.int/wftp3/av-arch/jvet-site/bitstream_exchange/VVCMultilayer/under_test/VTM-20.2/ shortly.
• In total currently 7 additional conformance bitstreams for VVC multilayer configurations had been generated.

HEVC Multilayer supporting extended bit depth activities:

• A merge request (MR) implementing the HEVC Multiview Main 10 profiles in JVET-AA1011 has been submitted in https://vcgit.hhi.fraunhofer.de/jvet/HTM/-/merge_requests/5
• An implementation for the other HEVC Multiview profiles in JVET-AA1011 (Multiview Monochrome, and Multiview Monochrome 10 profiles) is still in progress.
• Cross-checking of the 4 HEVC Multilayer supporting extended bit depth bitstreams in AD0232 is contingent to the merging of the above MR. Review of the MR is on-going.

The regular JVET e-mail reflector was used for discussions (jvet@lists.rwth-aachen.de).

The AHG5 chairs and JVET chairs can be reached at jvet-conformance@lists.rwth-aachen.de. Participants should not subscribe to this list but may send emails to it.

No contributions on the subject of this AHG were noted.

Ftp site information

The procedure to exchange the bitstream (ftp cite, bitstream files, etc.) is specified in Sec 2 “Procedure” of JVET-R2008. The ftp and http sites for downloading bitstreams are

• VVC:

ftp://ftp3.itu.int/jvet-site/bitstream_exchange/VVC

https://www.itu.int/wftp3/av-arch/jvet-site/bitstream_exchange/VVC/

• VVC operation range extensions:

ftp://ftp3.itu.int/jvet-site/bitstream_exchange/VVCv2

https://www.itu.int/wftp3/av-arch/jvet-site/bitstream_exchange/VVCv2

The ftp site for uploading bitstream file is as follows.

ftp://ftp3.itu.int/jvet-site/dropbox/

(user id: avguest, passwd: Avguest201007)

If using FileZilla, the following configuration is suggested:

In the Filezilla Edit 🡪 Settings 🡪 Connection menu, it may also be necessary to set the minimum TLS level to 1.0.

Recommendations

The AHG recommended the following:

• Proceed with the generation, cross-checking, and documentation of the additional conformance bitstreams for VVC, in particular conformance bitstreams for VVC multilayer configurations (JVET-AE2028).
• Maintain and update the conformance bitstream database and contribute to report problems in JVET document 1004.
• Continue the generation, cross-checking, and documentation of the conformance streams for the HEVC multiview profiles supporting extended bit depth (JVET-AA1011).

It was requested to clarify whether additional work can be expected for defining more bitstreams in the category of VVC multi-layer, or if at some point the verified bitstreams could be submitted as an extension of VVC conformance.