JVET-AE0003 JVET AHG report: Test model software development (AHG3) [F. Bossen, X. Li, K. Sühring (co-chairs), E. François, Y. He, K. Sharman, V. Seregin, A. Tourapis (vice-chairs)]
The software model versions prior to the start of the meeting were:
- VTM 21.0 (Jul. 2023)
- HM-18.0 (Apr. 2023)
- HM-16.21+SCM-8.8 (Mar. 2020)
- SHM 12.4 (Jan. 2018)
- HTM 16.3 (Jul. 2018)
- JM 19.1 (Apr. 2023)
- JSVM 9.19.15
- JMVC 8.5
- 3DV ATM 15.0 (no version history)
- HDRTools 0.24 (March 2023)
Software for MFC and MFCD is only available as published by ITU-T and ISO/IEC. It is planned to create repositories with the latest versions available in ITU-T H.264.2 (02/2016). All development history is lost.
Software development
Development was continued on the GitLab server, which allows participants to register accounts and use a distributed development workflow based on git.
The server is located at:
https://vcgit.hhi.fraunhofer.de
The registration and development workflow are documented at:
https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/wikis/VVC-Software-Development-Workflow
Although the development process is described in the context of the VTM software, it can be applied to all other software projects hosted on the GitLab server as well.
VTM related activities
The VTM software can be found at
https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/
The software development continued on the GitLab server. VTM versions 20.1 and 20.2 were tagged on May 8, and VTM version 21.0 was tagged on Jul. 9. VTM 21.1 is expected during the 31st JVET meeting.
VTM 20.1 was tagged on May 8, 2023. Changes include:
- Undefine macro from HDRTools
- Fix build when GDR_ENABLED is 0
- Some fixes after !2541 (Use strong-typed enum for chroma format)
- Fix parsing of SDI SEI message
- Fix parsing of MAI SEI message
- Fix parsing of multiple SEI payloads in one NAL unit
- Add command line option to print per-frame encoding time in float format
- Fix #1595: Set ptl_multi_layer_enabled_flag when more than one layer is encoded
- Rollback SEI bits computation as it is erroneous
- Fix compilation when EXTENSION_360_VIDEO is enabled
- Shorten Neural Network Post Filter Characteristics SEI parameters
- Fix type of literal in assert expression
- Fixes for multilayer coding
- Port changes from JVET-Z0072 to performance low delay configuration file
- Revert !2573 and add MultiLayerEnabledFlag=1 to two_layers.cfg
VTM 20.2 was tagged on May 8, 2023. Changes include:
- Remove macros from previous cycle
VTM 21.0 was tagged Jul. 9, 2023. Changes include:
- Fix #1599: include frame rate info in bitstream when provided in y4m file
- JVET-AD0169: Downsampling filters in range 1.1 to 1.35 based on Kaiser(7) windowed sinc
- JVET-AD0386: Processing Order SEI message update
- Fix #1598: signal chroma location type when input is y4m
- Fix #1597 nnpfc_constant_patch_size_flag and nnpfc_overlap and nnpfc_overlap parsing.
- JVET-AD0045: Encoder MV selections and DMVR revisited
- Avoid seg fault Slice::constructRefPicList
- Code cleanup: fix variable names, use proper data types, etc.
- Replace EncAppCfg::OptionalValue with std::optional
- Remove JVET_O0549_ENCODER_ONLY_FILTER_POL macro
- Fix output frame rate when using --MaxTemporalLayer in decoder
- Rename df::program_options_lite as ProgramOptionsLite
- Rename and initialize m_RCCpbSaturationEnabled
- JVET-AD0056 item 5: don't signal nnpfc_input_pic_output_flag[i] flags when nnpfc_num_input_pics_minus1==0
- Fix variable names in VideoIOYuv
- Fix parsing of frame rate parameter
- JVET-AD0067 item2: Swap signalling order for nnpfc_aux_inp_idc and nnpfc_inp_order_idc
- JVET-AD0129: Fix padding
- JVET-AD0054: nnpfc_absent_input_pic_zero_flag
- JVET-AD0141: NNPFA is associated with picture that is an output picture
- JVET-AD0383: Signalling scaling ratio for output picture size instead of absolute width and height
- JVET-AD0067 Item 7: Include syntax element nnpfc_full_range_flag
- Fix #1596 on encoding crash when DeltaQpRD is used
- Avoid CU copy in EncModeCtrlMTnoRQT::useModeResult
- JVET-AD0056 Item 4: picture width and height constraints
- Fix default values of SEI NNFPC parameters to avoid crash and fix variable names
- Check config parameters after they have been set
- JVET-AD0056 Item 6: move nnpfc_inp_order_idc and nnpfc_out_order_idc
- JVET-AD0056 Item 1: move nnpfc_base_flag next to nnpfc_id
- JVET-AD0056 Item 11: nnpfc SEI metadata extension
- JVET-AD0056 Item 9: add condition for nnpfc padding signalling
- Fix compile issues with GCC 13 on Windows
- Remove singleComp function and clean up singleChan function
- JVET-AD0056 Item 7: nnpfc_inp_order_idc and nnpfc_out_order_idc constraints
- JVET-AD0056 Item 8: Move nnpfc_separate_colour_description_present_flag
- Add CHECK statement to avoid possible buffer overflow
- fix MSAN issues
- JVET-AD0091 items 4-6: add constraints for nnfpc padding
- Fix RPL construction
- JVET-AD0056 Item 12: remove redundant constraints on nnpfa_target_id
- JVET-AD0233 Items 1,2,6: Add NNFPC constraints and chroma sample location
- JVET-AD0056 Item 13: implementation of nnpfa_target_base_flag
- Fix #1603 Overflow in importMergeInfo()
- JVET-AD0057 item 6: multi-picture-input constraint
- Fix #1604: NNPFC constraints check
- JVET-AD0057 items 3 and 4: Update VclAssociatedSeiList and PicUnitRepConSeiList
- JVET-AD0057 item 2: disallow the post-filter hint SEI message to be present in both a prefix SEI NALU and a suffix SEI NALU in the same picture unit
- JVET-AD0388: add nnfpa_num_output_entries and nnfpa_output_flag syntax elements
VTM 21.1 is expected to be tagged during the 31st JVET meeting. Changes are expected to include the remaining implementations of the 30th meeting.
CTC Performance
The following tables shows VTM 20.0 performance over HM 17.0 (not updated).
All Intra Main 10 | |||||
Over HM-17.0 | |||||
Y | U | V | EncT | DecT | |
Class A1 | -29.50% | -32.81% | -33.83% | 1273% | 166% |
Class A2 | -29.73% | -24.39% | -21.58% | 2142% | 174% |
Class B | -22.32% | -27.21% | -30.98% | 2397% | 175% |
Class C | -22.89% | -19.53% | -23.19% | 3348% | 179% |
Class E | -26.04% | -25.90% | -24.15% | 1869% | 158% |
Overall | -25.50% | -25.75% | -27.02% | 2187% | 171% |
Class D | -18.80% | -13.85% | -13.68% | 4270% | 189% |
Class F | -39.49% | -40.23% | -42.90% | 4220% | 171% |
Random access Main 10 | |||||
Over HM-17.0 | |||||
Y | U | V | EncT | DecT | |
Class A1 | -40.60% | -40.71% | -47.16% | 586% | 158% |
Class A2 | -44.02% | -41.73% | -40.78% | 679% | 170% |
Class B | -37.41% | -50.12% | -48.51% | 672% | 161% |
Class C | -33.86% | -36.28% | -38.21% | 923% | 164% |
Class E | |||||
Overall | -38.42% | -42.87% | -43.95% | 713% | 163% |
Class D | -31.72% | -32.60% | -31.95% | 970% | 161% |
Class F | -46.07% | -49.76% | -50.65% | 490% | 145% |
Low delay B Main 10 | |||||
Over HM-17.0 | |||||
Y | U | V | EncT | DecT | |
Class A1 | |||||
Class A2 | |||||
Class B | -30.40% | -37.43% | -35.12% | 629% | 152% |
Class C | -28.18% | -21.23% | -21.06% | 817% | 167% |
Class E | -31.16% | -35.72% | -28.79% | 309% | 140% |
Overall | -29.85% | -31.60% | -28.85% | 575% | 154% |
Class D | -26.75% | -16.50% | -14.88% | 881% | 179% |
Class F | -42.17% | -44.02% | -44.14% | 435% | 137% |
Low delay P Main 10 | |||||
Over HM-17.0 | |||||
Y | U | V | EncT | DecT | |
Class A1 | |||||
Class A2 | |||||
Class B | -34.52% | -38.34% | -36.12% | 526% | 160% |
Class C | -28.07% | -17.73% | -18.09% | 670% | 176% |
Class E | -33.06% | -37.89% | -31.10% | 280% | 146% |
Overall | -32.01% | -31.36% | -28.85% | 487% | 161% |
Class D | -26.68% | -12.73% | -11.47% | 762% | 175% |
Class F | -40.13% | -41.24% | -41.72% | 437% | 145% |
According to common test conditions in random access configuration HM is using a GOP size of 16 pictures compared to VTM using a GOP of 32 pictures. Random access points are inserted approximately every second aligned with a GOP boundary of GOP 32 in both VTM and HM. The VTM uses two more reference pictures in random access than HM (due to more memory being available in typical level settings).
There was no change in coding performance or run time between VTM 21.0 and VTM 20.1 using SDR CTC.
HDR and high bit depth CTC were stated. Partial results show no performance change as well.
The following tables show VTM 20.0 performance over HM 17.0 using HDR CTC (not updated):
Random Access | ||||||||||
Over HM17.0 | ||||||||||
wPSNR | PSNR | |||||||||
DE100 | PSNR-L100 | Y | U | V | Y | U | V | EncT | DecT | |
Class H1 | -39.54% | -37.56% | -36.67% | -54.98% | -48.45% | -33.68% | -49.48% | -40.84% | 286% | 97% |
Class H2 | -31.97% | -57.69% | -63.36% | 271% | 97% | |||||
Overall | -39.54% | -37.56% | -36.67% | -54.98% | -48.45% | -33.06% | -52.47% | -49.03% | 280% | 97% |
All Intra | ||||||||||
Over HM17.0 | ||||||||||
wPSNR | PSNR | |||||||||
DE100 | PSNR-L100 | Y | U | V | Y | U | V | EncT | DecT | |
Class H1 | -41.45% | -27.31% | -26.79% | -57.74% | -52.79% | -24.00% | -52.93% | -45.13% | 1538% | 113% |
Class H2 | -21.76% | -47.21% | -50.57% | 1320% | 109% | |||||
Overall | -41.45% | -27.31% | -26.79% | -57.74% | -52.79% | -23.18% | -50.85% | -47.11% | 1455% | 112% |
Issues in VTM affecting conformance
The following issues in VTM master branch may affect conformance:
- Missing HLS features (see sections below)
Status of implementation of proposals of previous JVET meetings
The following list contains all adoptions of the Q and R meetings that were not marked as merged (or submitted) or specification only change in the software coordinator tracking sheet:
- JVET-Q0112
- JVET-Q0154: Disallow mixing of GDR and IRAP (Disallow mixing of GDR with any non-GDR).
- JVET-Q0164
- JVET-Q0402
- JVET-R0178: Require that when no_aps_constraint_flag is equal to 1, sps_lmcs_enabled_flag and sps_scaling_list_enabled_flag shall be equal to 0
- JVET-R0221
- JVET-R0046: Change the description of the bitstream extraction process per the value of max_tid_il_ref_pics_plus1[ ][ ] (aspect 1.2 per JVET-R0046-v4).
- JVET-R0065: Specify that GDR AUs shall be complete – i.e., all of the layers in the CVS shall have a picture in the AU (as with IRAP AUs).
- JVET-R0191: Update the range value for num_ols_hrd_params_minus1.
- JVET-R0222 aspect 1: Infer vps_max_sublayers_minus1 to be equal to 6 when sps_video_parameter_set_id is equal to 0 (i.e. VPS is not present). The exact editorial expression is at the discretion of the editor.
- JVET-S0196 (JVET-S0144 item 17)
- JVET-S0227 (JVET-S0144 item 22)
- JVET-S0077 (JVET-S0139 item 5)
- JVET-S0174 aspect 2 (JVET-S0139 item 18.b)
- JVET-S0156 aspect 3 (JVET-S0139 item 21)
- JVET-S0139 item 26 (no source listed, text only?)
- JVET-S0188 aspect 1 (JVET-S0139 item 28)
- JVET-S0139 item 40 (item does not exist)
- JVET-S0042 (JVET-S0142 item 1.b)
- JVET-S0174 aspect 1 (JVET S0143 item 19)
- JVET-S0096 aspect 3 (JVET-S0140 item 10)
- JVET-S0096 aspect 4 (JVET-S0140 item 13)
- JVET-S0159 aspect 3 (JVET-S0140 item 16)
- JVET-S0171 (JVET-S0256)
- JVET-S0118 (JVET-S0141 item 7)
- JVET-S0102 (JVET-S0141 item 9.a)
- JVET-S0157 item 2 (JVET-S0141 item 13)
- JVET-S0157 item 4 (JVET-S0141 item 14)
- JVET-S0175 aspect 3 (JVET-S0141 item 16)
- JVET-S0175 aspect 1, 2 (JVET-S0141 item 17)
- JVET-S0175 aspects 4 and 5 (JVET-S0141 item 18)
- JVET-S0175 aspect 6 (JVET-S0141 item 19)
- JVET-S0198/ JVET-S0223 (JVET-S0141 item 24)
- JVET-S0173 aspect 2 (JVET-S0141 item 40.b)
- JVET-S0173 item 1 (JVET-S0141 item 51)
- JVET-S0173 item 3 (JVET-S0141 item 52)
- JVET-S0173 item 5 (JVET-S0141 item 53)
- JVET-S0173 item 6 (JVET-S0141 item 54)
- JVET-S0173 item 4 (JVET-S0141 item 56)
- JVET-S0176 item 4 (JVET-S0141 item 60)
- JVET-S0154 aspect 5 (JVET-S0141 item 68)
- JVET-S0154 aspect 6 (JVET-S0141 item 69)
- JVET-S0154 aspect 8 (JVET-S0141 item 71)
- JVET-S0095 aspect 5 (JVET-S0145 item 5)
- JVET-S0095 aspect 6 (JVET-S0145 item 6)
- JVET-S0100 aspect 1, depends on JVET-R0193 (JVET-S0147 item 2)
- FINB ballot comments
- Make high tier support up to 960.
HM related activities
HM 18.0 was tagged on Apr. 20, 2023. Changes include:
- Update to Ubuntu 22.04 build server
- Update the HDR configuration files to align with the HDR CTC document (JVET-AC2011)
- Update to FilmGrainCharacteristics SEI support, including applying film grain noise onto the decoded output.
The performance of HM 18.0 is identical to HM 17.0 under SDR and HDR CTCs. Initial results also indicate that the performance is identical to HM 17.0 under RExt non-4:2:0 and RExt high bit depth CTCs.
The following MRs are pending [with status indicated]:
- Port the Y4M support [one issue remains]
- Mark the current picture as short-term ref (for SCM) [need SCC expert reviewer]
The HM SCC (SCM) branch (HM-16.21+SCM-8.8) has not been updated for the recent HM versions. Updating SCM to, for example, HM-18.0+SCM-8.8 should be considered. It may though be helpful to move SCC related functionality into separate source files. Volunteer work towards merging the branches would be appreciated.
As reported in the previous reports, further information on lambda optimisation in HM would be appreciated, including comparison of allocation of bits within the GOP structures between HM and VTM.
Two newer tickets were filed for HM in relation to 360Lib and the NoOutPrior_A_Qualcomm_1 conformance stream.
Otherwise the HEVC bug tracker lists:
- 43 tickets for “HM”, most of which are more than 5 years,
- 1 ticket for “HM RExt”,
- 9 tickets for “HM SCC”, most of which are at least 3 years old,
Help to address these tickets would be appreciated.
One merge request (MR#40) is available related to HM SCC for ticket #1511 on SCC reference picture marking. Help was requested to confirm that the proposed change matches the SCC text.
360Lib related activities
The latest 360Lib software can be found at https://vcgit.hhi.fraunhofer.de/jvet/360lib
There had not been any further developments to 360Lib during this meeting cycle.
SCM related activities
There had not been any further developments to SCC’s SCM during this meeting cycle.
SHM related activities
There had not been any further developments to SHVC’s SHM during this meeting cycle.
HTM related activities
There had not been any releases of HTM of MV-HEVC and 3D-HEVC.
The next release will include the following changes:
- JVET-Z0209: Early termination during calculating RDcost of depth
HDRTools related activities
There had not been any further developments to HDRTools during this meeting cycle.
JM, JSVM, JMVM related activities
There had not been any further developments to JM, JSVM or JMVM during this meeting cycle.
For the previous release of the JM, only basic testing was performed to confirm that the code still works, but performance testing was pending. Performance changes are unlikely but may occur due to bug fixes. A possible encoder/decoder mismatch had been reported privately, but needed more investigation.
Bug tracking
The bug tracker for VTM and specification text is located at:
https://jvet.hhi.fraunhofer.de/trac/vvc
The bug tracker uses the same accounts as the HM software bug tracker. Users may need to log in again due to the different sub-domain. For spam fighting reasons account registration is only possible at the HM software bug tracker at
https://hevc.hhi.fraunhofer.de/trac/hevc
Bug tracking for HDRTools is located at:
https://gitlab.com/standards/HDRTools/-/issues
Contributors are requested to please file all issues related to the VVC reference software and HDRTools into the appropriate bug tracker. Try to provide all the details, which are necessary to reproduce the issue. Patches for solving issues and improving the software are always appreciated.
Software repositories
Git repositories that were previously assigned to the JCT-VC group on the GitLab server were re-assigned to the JVET group. The old URLs are still working and will forward the user to the new location, with the display of a warning suggesting to update bookmarks to the new location. The SVN repository for 360Lib was converted to git and development was moved to the GitLab server. Historical branches can still be accessed in the SVN repository.
CTC alignment and merging
There are currently 8 JVET CTC documents: