JVET-AC0003 JVET AHG report: Test model software development (AHG3) [F. Bossen, X. Li, K. Sühring (co-chairs), E. François, Y. He, K. Sharman, V. Seregin, A. Tourapis (vice-chairs)]

The software model versions prior to the start of the meeting were:

• VTM 19.0 (Nov. 2022)
• HM-17.0 (Jan. 2023)
• HM-16.21+SCM-8.8 (Mar. 2020)
• SHM 12.4 (Jan. 2018)
• HTM 16.3 (Jul. 2018)
• JM 19.0
• JSVM 9.19.15
• JMVC 8.5
• 3DV ATM 15.0 (no version history)
• HDRTools 0.23 (October 2021)

Software for MFC and MFCD is only available as published by ITU-T and ISO/IEC. It is planned to create repositories with the latest versions available in ITU-T H.264.2 (02/2016). All development history is lost.

Software development

Development was continued on the GitLab server, which allows participants to register accounts and use a distributed development workflow based on git.

The server is located at:

https://vcgit.hhi.fraunhofer.de

The registration and development workflow are documented at:

https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/wikis/VVC-Software-Development-Workflow

Although the development process is described in the context of the VTM software, it can be applied to all other software projects hosted on the GitLab server as well.

VTM related activities

The VTM software can be found at

https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/

The software development continued on the GitLab server. VTM versions 18.1 and 18.2 were tagged on Oct. 24 and Oct. 25, and VTM version 19.0 was tagged on Nov. 30. VTM 19.1 is expected during the 29^th JVET meeting.

VTM 18.1 was tagged on Oct. 24, 2022. Changes include:

• Fix memory leaks
• Add the missing setting of temporal filter to RA GOP16 cfg
• fix active picture number for L0 in reference picture list creation
• Fix #1574: SPS ID and PPS ID in encoding writeoutput()
• Add filler payload SEI message
• Fix RPL-based marking and RPL checks in Multilayer context
• Fix a Y4M bug (chroma scaling was not considered)
• fix reference picture lists checking for multilayer scalable
• Use bool constants and operators where appropriate
• Fix variable names
• Fix #1576: use correct interpolation filter size
• Fix DeblockingFilterDisable = 1
• Fix variable name and compile issue on Xcode
• Fix compile error with tracing is enabled
• Fix indentation and braces
• Fix indentation and braces
• Fix #1575: Use per-layer APS ids range for ALF APSs
• Avoid compile issue when using address sanitizer
• Remove JVET_X0143_ALF_APS_ID_OFFSET related code
• Fix GDR code to avoid invalid reference
• Fix #1580: L1 RPL filling for incomplete GOP
• Fix: avoid redundant coding of RPL in both SPS and SH in multilayer
• Add QPIncrementFrame description in SW manual

VTM 18.2 was tagged on Oct. 25, 2022. Changes include:

• Remove macros from previous cycle

VTM 19.0 was tagged Nov. 30, 2022. Changes include:

• JVET-AB0085: Support multiple PTL signalling (encoder only)
• JVET-AB0047: Moving the gated syntax that contains nnpfc_uri_tag and nnpfc_uri...
• JVET-AB0080: GOP-based RPR encoder control also including a fix for chroma QP
• JVET-AB0081: Increased length of filters used for upscaling reconstructed pictures
• JVET-AB0072: VTM Encoder Implementation for Green-MPEG SEI Messaging
• Fix which conformance window is used in case of field coding
• Clean up code related to sign derivation
• Fix variable names
• Fix which PTL is used for checks
• Fix trailing white space, variable names, braces
• Fix variable names and spacing
• Fix variable name
• Fix inference of bit-depth for YUV files
• Fix crash during checks for layer ID of ALF APS
• Use static_vector instead of std::vector for list of subprofiles
• Remove default parameter values and fix braces/indentation
• Use vector object to hold frame-level delta qps
• Avoid code duplication and repeated memory allocation
• Combine redundant ALF constants m_NUM_BITS and m_scaleBits
• Reduce number of calls to Rice parameter derivation
• Use strong-typed enum for BPMType
• Clean up Analyze class and related code
• Updated formatRGB.cfg to include RGBtoGBR conversion
• Address compiler warnings/errors
• Clean up adaptive BT size related code
• Fix variable names
• Clean up GPM encoder code
• Use named constant for maximum CU size for IBC
• Fix !2364: distinguish number of SAD and SATD candidates
• Clean up BDOF condition checking
• Fix typo in function name
• Use bool type and constants for boolean fields
• Fix variable names and indentations
• Remove default parameter values for printOut function
• Use static_vector instead of custom structures with separate array and item count
• Use strong-typed enum for TransType
• Use strong-typed enum for HashType
• Clean up handling of ALF data in Picture object
• Use named constants for CBF masks
• Clean up SAO code and use strong typed enums
• Remove most default parameters from motion compensation functions
• Clean up matrix multiplication functions
• Clean up HMVP related functions
• Setup general HRD parameters
• Use strong-typed enum for CoeffScanType
• Clean up ComprCUCtx data structure
• Rename JVET_C0024_ZERO_OUT_TH and clean up related functions
• Clean up deblocking filter code
• Remove -Wno-class-memaccess compiler option
• Don't optimize for deblocking when deblocking is disabled
• Clean up applyDeblockingFilterParameterSelection()
• Clean up checking slice/tile/subpic conditions in deblocking filter
• Clean up filter length handling in deblocking filter
• Clean up edge/bs handling in deblocking filter
• Avoid double braces for init
• Use enumeration for BDPCM mode
• Clean up motion discontinuity processing in deblocking filter
• Define type to hold list of ALF APS Ids and clean up related code
• Allow GREEN_METADATA_SEI_ENABLED to be set externally
• Fix variable names
• Use proper preprocessor guards for GREEN_METADATA_SEI_ENABLED
• Avoid using UNDEFINED which is defined as a macro by HDRLib
• Fix: add check that was removed during integration
• Fix SEIBufferingPeriod::copyTo()
• Fix for zero-sized SEI NAL units
• Fix parsing of pt_sublayer_delays_present_flag
• Fix activateAPS() to update pointers to APSs
• Store latest DRAP POC in DecLib and set it to Slice
• Fix checkCRA() for field pictures

VTM 19.1 is expected to be tagged during the 29^th JVET meeting. Changes include so far:

• JVET-T0056: SEI manifest and SEI prefix indication
• JVET-AB0069 change the length of the po_sei_processing_order[i] syntax element
• JVET-AB0135 and BC#017 Add nnpfc_total_kilobyte_size and replace Remove unused functions
• M60678: ballot comments of FI_03
• JVET-AB0058: signal frame upsampling in neural network post-filter characteristics
• JVET-AB0051: Modification of SEI processing order SEI message
• JVET-AB0049: Modifications of NNPFC and NNPFA
• JVET-AB0050: Add two flags for NNPFA
• JVET-AB0070: post-filter hint SEI
• Use constant for subblock size in deblocking filter
• Fix variable names
• Define constant for VDPU size and fix function name
• Clean up intra and CIIP code
• Fix variable names
• Fixes in SubpicMergeApp for NumSubPics and SubPicId setters
• Use strong-typed enum for MtsType
• Clean deblocking filter code
• SubPicMergeApp: remove duplicate ceilLog2 function
• Clean up DMVR code
• Fix compilation when disabling REUSE_CU_RESULTS
• Use named constants
• Use named variable for unset cost
• Clean up reading/parsing of number of active refs
• Define named reference index for IBC
• Clean up ISP and use strong typed enum
• Use strong typed enum for SEI payload type
• Rename dynamic_cache to Pool
• nnpfc_complexity_idc with a flag
• Rename LFCUParam data structure
• Clean up MMVD related code
• Define RefSetArray template to define arrays indexed by listIdx and refIdx
• Move new macros into section where they are indicated to be removed in next cycle
• Fix compilation when JVET_R0351_HIGH_BIT_DEPTH_ENABLED is enabled
• Use python3 as interpreter for cmake/CMakeBuild/bin/cmake.py
• Use single variable to control ENABLE_TRACING
• Fix a gcc12 compiling warning/error
• Add option to enable RExt__HIGH_BIT_DEPTH_SUPPORT from command line
• Fix variable names
• Clean up hash related functions
• Remove Ubuntu 18.04 from auto-build pipeline
• Use ptrdiff_t for strides
• Use strong enum type for AffineModel
• Clean buffer usage in merge RD checking functions
• Clean up BCW and AMVR related code
• Clean up merge context data structure
• Use strong typed enum for MESearchMethod
• Remove unused field
• Use strong typed ChannelType
• Use strong typed enum for MergeType
• Fix gcc 12 compiling errors
• Replace m_subPuMC by a local variable
• Clean up MV precision handling functions and use strong-typed enum
• Define MergeIdxPair type to hold pair of merge indices
• Clean up inter merge related functions
• Clean up width/height in InterPrediction::xPredInterBlk
• Remove m_iRefListIdx field from InterPrediction class
• Use strong typed enum for APS type
• Remove unused variable
• Use proper data type for RPR scaling
• Fix variable names and use named constants
• Use strong-type enum for DFunc
• Change copyright year to 2023
• Remove unused field
• Fix parameter type in function definition: use ptrdiff_t instead of uint32_t
• Clean up MVD encoding
• Group partition contexts for faster copy
• Define enum for MIP size and use Pel elements for buffers
• Improve adherence to coding guidelines in EncGOP.cpp
• Improve adherence to coding guidelines
• Remove mvRefine field from InterPredictionData
• Fix macOS build with trace file enabled
• Use Size type instead of array of two values
• Remove g_tbMax and use floorLog2 instead
• Move sei_processing_order.cfg into sei_vui subdirectory
• Remove duplicate function and fix data types
• Add Ubuntu 22.04 auto-build
• Remove macros for HLS syntax element read/write

CTC Performance

The following tables show VTM 19.0 performance over HM 17.0.

According to common test conditions in random access configuration HM is using a GOP size of 16 pictures compared to VTM using a GOP of 32 pictures. Random access points are inserted approximately every second aligned with a GOP boundary of GOP 32 in both VTM and HM. VTM uses two more reference pictures in random access than HM (due to more memory being availably in typical level settings).

The following tables show VTM 19.0 performance compared to VTM 18.0 using SDR CTC:

The following tables shows VTM 19.0 performance over HM 17.0 using HDR CTC:

The following tables show VTM 19.0 performance compared to VTM 18.0 using HDR CTC:

Full results are attached to this AHG report as Excel files.

Issues in VTM affecting conformance

The following issues in VTM master branch may affect conformance:

• Missing HLS features (see sections below)

Merge request 2343 (“Fix which PTL is used for checks”) fixes which Profile/Tier/Level data structure is used (SPS or VPS) for some conformance checks. After this correction (i.e., VTM versions 19.0 and later), the following conformance streams fail to decode:

• ILRPL_A_Huawei_2
• OLS_A_Tencent_5
• OLS_B_Tencent_5
• OLS_C_Tencent_5
• OPI_B_Nokia_3
• SPATSCAL444_A_Qualcomm_2
• SPATSCAL_A_Qualcomm_3
• VPS_A_INTEL_3
• VPS_B_ERICSSON_1
• VPS_C_ERICSSON_2

Status of implementation of proposals of previous JVET meetings

The following list contains all adoptions of the Q and R meetings that were not marked as merged (or submitted) or specification only change in the software coordinator tracking sheet:

• JVET-Q0112
• JVET-Q0154: Disallow mixing of GDR and IRAP (Disallow mixing of GDR with any non-GDR).
• JVET-Q0164
• JVET-Q0402
• JVET-R0178: Require that when no_aps_constraint_flag is equal to 1, sps_lmcs_enabled_flag and sps_scaling_list_enabled_flag shall be equal to 0
• JVET-R0221
• JVET-R0046: Change the description of the bitstream extraction process per the value of max_tid_il_ref_pics_plus1[ ][ ] (aspect 1.2 per JVET-R0046-v4).
• JVET-R0065: Specify that GDR AUs shall be complete – i.e., all of the layers in the CVS shall have a picture in the AU (as with IRAP AUs).
• JVET-R0191: Update the range value for num_ols_hrd_params_minus1.
• JVET-R0222 aspect 1: Infer vps_max_sublayers_minus1 to be equal to 6 when sps_video_parameter_set_id is equal to 0 (i.e. VPS is not present). The exact editorial expression is at the discretion of the editor.
• JVET-S0196 (JVET-S0144 item 17)
• JVET-S0227 (JVET-S0144 item 22)
• JVET-S0077 (JVET-S0139 item 5)
• JVET-S0174 aspect 2 (JVET-S0139 item 18.b)
• JVET-S0156 aspect 3 (JVET-S0139 item 21)
• JVET-S0139 item 26 (no source listed, text only?)
• JVET-S0188 aspect 1 (JVET-S0139 item 28)
• JVET-S0139 item 40 (item does not exist)
• JVET-S0042 (JVET-S0142 item 1.b)
• JVET-S0174 aspect 1 (JVET S0143 item 19)
• JVET-S0096 aspect 3 (JVET-S0140 item 10)
• JVET-S0096 aspect 4 (JVET-S0140 item 13)
• JVET-S0159 aspect 3 (JVET-S0140 item 16)
• JVET-S0171 (JVET-S0256)
• JVET-S0118 (JVET-S0141 item 7)
• JVET-S0102 (JVET-S0141 item 9.a)
• JVET-S0157 item 2 (JVET-S0141 item 13)
• JVET-S0157 item 4 (JVET-S0141 item 14)
• JVET-S0175 aspect 3 (JVET-S0141 item 16)
• JVET-S0175 aspect 1, 2 (JVET-S0141 item 17)
• JVET-S0175 aspects 4 and 5 (JVET-S0141 item 18)
• JVET-S0175 aspect 6 (JVET-S0141 item 19)
• JVET-S0198/ JVET-S0223 (JVET-S0141 item 24)
• JVET-S0173 aspect 2 (JVET-S0141 item 40.b)
• JVET-S0173 item 1 (JVET-S0141 item 51)
• JVET-S0173 item 3 (JVET-S0141 item 52)
• JVET-S0173 item 5 (JVET-S0141 item 53)
• JVET-S0173 item 6 (JVET-S0141 item 54)
• JVET-S0173 item 4 (JVET-S0141 item 56)
• JVET-S0176 item 4 (JVET-S0141 item 60)
• JVET-S0154 aspect 5 (JVET-S0141 item 68)
• JVET-S0154 aspect 6 (JVET-S0141 item 69)
• JVET-S0154 aspect 8 (JVET-S0141 item 71)
• JVET-S0095 aspect 5 (JVET-S0145 item 5)
• JVET-S0095 aspect 6 (JVET-S0145 item 6)
• JVET-S0100 aspect 1, depends on JVET-R0193 (JVET-S0147 item 2)
• FINB ballot comments
• Make high tier support up to 960.

HM related activities

HM 17.0 was tagged on Jan. 5, 2023. Changes include:

• JVET-AA0130: VTM and HM common test conditions for high bit depth
• JVET-AB0271: per-sequence config files for HDR content
• JCTVC-AD0021(JVET-T0056) SEI manifest & SEI prefix indication
• Build fixes for Xcode 14.2
• Remove Ubuntu 18.04 from auto-build pipeline
• Change Python executable to python3 for build environment scripts

The following tables show HM 17.0 performance compared to HM 16.26 under SRD CTC:

The following table shows HM 17.0 performance compared to HM 16.26 under HDR CTC:

The following MRs were reported as pending [with status indicated]:

• JVET-X0048: implementation of film grain characteristics (FGC) SEI message [waiting for proponent feedback]
• Port the Y4M support [one issue remaining]
• Mark the current picture as short-term ref (for SCM) [need SCC expert reviewer]

The HM SCC (SCM) branch (HM-16.21+SCM-8.8) had not been updated for the recent HM versions. Updating SCM to, for example, HM-17.0+SCM-8.8 should be considered. It may though be helpful to move SCC related functionality into separate source files. Volunteer work towards merging the branches would be appreciated.

As reported in the previous reports, further information on lambda optimization in HM would be appreciated, including comparison of allocation of bits within the GOP structures between HM and VTM.

The HEVC bug tracker listed:

• 38 tickets for “HM”, most of which are more than 5 years old,
• 1 ticket for “HM RExt”, which was created during this reporting period,
• 7 tickets for “HM SCC”, all of which are at least 3 years old,

Help to address these tickets would be appreciated.

One merge request is available related to HM SCC for ticket #1511 on SCC reference picture marking. We would appreciate help to confirm that the proposed change matches the SCC text.

360Lib related activities

The latest 360Lib software (360Lib-13.4) can be found at https://vcgit.hhi.fraunhofer.de/jvet/360lib/-/tags/360Lib-13.4

The following table is for the projection formats comparison using VTM-19.0 according to 360-degree video CTC (JVET-U2012) compared to that using VTM-18.0 (VTM-18.0 as anchor).

The following table compares generalized cubemap (GCMP) coding and padded equi-rectangular projection (PERP) coding using VTM-19.0 (PERP as anchor).

The following tables are for PERP and GCMP coding comparison between VTM-19.0 and HM-16.22 (HM as anchor), respectively.

SCM related activities

There had not been any further developments to SCC’s SCM during this meeting cycle.

SHM related activities

There had not been any further developments to SHVC’s SHM during this meeting cycle.

HTM related activities

There had not been any releases of HTM of MV-HEVC and 3D-HEVC.

The next release will include the following changes:

• JVET-Z0209: Early termination during calculating RDcost of depth

HDRTools related activities

There had not been any updates of HDRTools.

New development is being added under the branch named 0.24-dev.

JM, JSVM, JMVM related activities

There had not been any updates to the JM, JSVM and JMVM software.

1. Bug tracking

The bug tracker for VTM and specification text is located at:

https://jvet.hhi.fraunhofer.de/trac/vvc

The bug tracker uses the same accounts as the HM software bug tracker. Users may need to log in again due to the different sub-domain. For spam fighting reasons account registration is only possible at the HM software bug tracker at

https://hevc.hhi.fraunhofer.de/trac/hevc

Bug tracking for HDRTools is located at:

https://gitlab.com/standards/HDRTools/-/issues

Please file all issues related to the VVC reference software and HDRTools into the appropriate bug tracker. Try to provide all the details, which are necessary to reproduce the issue. Patches for solving issues and improving the software are always appreciated.

1. Software repositories

Git repositories that were previously assigned to the JCT-VC group on the GitLab server were re-assigned to the JVET group. The old URLs are still working and will forward the user to the new location, with the display of a warning suggesting to update bookmarks to the new location. The SVN repository for 360Lib was converted to git and development was moved to the GitLab server. Historical branches can still be accessed in the SVN repository.

CTC alignment and merging

Following the merger for high-bit depth CTCs (JVET-AA0130), there were currently 7 related JVET CTC documents, namely: