JVET-Y0005 JVET AHG report: Conformance testing (AHG5) [D. Rusanovskyy, I. Moccagatta, F. Bossen, K. Kawamura, T. Hashimoto, H.-J. Jhu, K. Sühring, Y. Yu]
Timeline
The timeline of progress on the Conformance testing specification is as follows:
- 24th meeting Oct 2021:
- FDIS v1 conformance
- CDAM v2 conformance
- 25th meeting Jan 2022:
- ITU-T Consent for v1
- DAM 1 v2 ballot in ISO/IEC
- 27th meeting July 2022: FDAM v2
- 28 th meeting October 2022 ITU consent for v2
Status on bitstream submission
The status at the time of preparation of this report is as follows:
- conformance bitstreams for VVC:
- 104 bitstream categories have been identified
- At least one bitstream has been submitted in each identified category
- 283 total bitstreams have been provided, checked, and made available
- conformance bitstreams for VVC operation range extensions:
- 51 bitstream categories have been identified
- Volunteers have been identified to generate bitstreams in all categories
- Volunteers have been identified to cross-check bitstreams in 47 categories
- 73 bitstream descriptions have been provided
- 102 bitstreams of 47 identified categories have been cross-checked and uploaded
- 20 bitstreams of 4 identified categories have been generated and are waiting to be cross-checked
Activities and Discussion
The AHG activities are on schedule with the preliminary timeline shown in section 2.
VVC activities:
- Three bitstreams (CROP_A_Panasonic, TEMPSCAL_B_Panasonic, and TEMPSCAL_C_Panasonic) had been re-generated with the sequence BQMall because of potential licensing issue of the sequence RitualDance.
There were not currently any known issues with the other provided VVC conformance bitstream packages. A VTM15 directory of bitstreams was made available. All provided bitstreams can be decoded using VTM15.
VVC operation range extensions activities:
- Volunteers to generate and cross-check the bitstreams have been solicited and identified.
- Volunteers and cross-checkers have exchanged bitstreams using their own means, and bitstreams have been uploaded when cross-check has been confirmed.
- All cross-checked bitstreams can be decoded using VTM14.
- Most of the cross-checked bitstreams cannot be decoded using VTM15 because of this error
ERROR: In function "parseConstraintInfo" in VVCSoftware_VTM/source/Lib/DecoderLib/VLCReader.cpp:4638: gci_reserved_zero_bit not equal to zero
This problem seems related to the bug reported in JVET-Y0190:
JVET-Y0190 abstract: It is asserted that the current text for general constraints information (GCI) in VVC version 2 contains two bugs. For the first bug, it is asserted that incorrect semantics and handling of gci_num_additional_bits lead to GCI decoding behaviour in VVC version 2 incompatible with VVC version 1 decoders. For the second bug, it is asserted that setting gci_present_flag to 1 and gci_num_additional_bits to 0 leads to undefined constraint behaviour for the 6 coding tools which are governed by the 6 additional GCI flags introduced in VVC version 2.
- Some packages have minor issues (i.e., file names do not match package name, incorrect format of MD5, etc.). Plan is to fix them when re-generating the streams following resolution of VTM15.0 error.
The regular JVET e-mail reflector was used for discussions (jvet@lists.rwth-aachen.de).
The AHG5 chairs and JVET chairs can be reached at jvet-conformance@lists.rwth-aachen.de. Participants should not subscribe to this list but may send emails to it.
Contributions
Two input contributions were noted as relevant: