JVET-AD0006 JVET AHG report: ECM software development (AHG6) [V. Seregin (chair), J. Chen, F. Le Léannec, K. Zhang (vice-chairs)]
Software development
ECM software repository is located at https://vcgit.hhi.fraunhofer.de/ecm/ECM.
ECM software is based on VTM-10.0 with enabled MCTF including the update from JVET-V0056, and GOP32, which is very close to VTM-11.0.
VTM-11.0ecm anchor https://vcgit.hhi.fraunhofer.de/ecm/ECM/-/tree/VTM11_ANC is used for ECM performance evaluation.
The following adopted aspects were integrated into ECM-7.0:
- JVET-AB0171: Asymmetric deblocking (MR 303)
- JVET-AC0105: Directional planar (MR 306)
- JVET-AC0095: fix padding at picture boundary (MR 312)
- JVET-AC0189: SGPM without blending (MR 313)
- JVET-AC0144: affine DMVR regression (MR 310)
- JVET-AC0147: Subsampling is not applied to CCCM (MR 314)
- JVET-AC0112: IBC-CIIP, IBC-GPM, IBC-LIC (MR 315)
- JVET-AC0053: Gaussian solver for filter coefficients (MR 318)
- JVET-AC0071: Direct block vector mode for chroma prediction (MR 325) (part of AC0113 3.5a)
- JVET-AC0130: NSPT replacing DCT-II + LFNST for certain block shapes (MR 319)
- JVET-AC0123: MTT correction for class B QP22 (MR 322)
- JVET-AC0158: Pixel based affine motion compensation (MR 327)
- JVET-AC0094: Optimizing the use of reference samples (MR 311)
- JVET-AC0185: Enhanced temporal motion information derivation (MR 317)
- JVET-AC0162: ALF using residual samples as additional inputs (MR 323)
- JVET-AC0335: Content adaptive OBMC enabling (MR 324)
- JVET-AC0119: On Chroma Fusion improvement (MR 331)
- JVET-AC0098: Location-dependent DIMD (MR 329)
- JVET-AC0054: Gradient and location-based CCCM (MR 332)
- JVET-AC0115: Intra mode derivation for IntraTMP transform (Test 4.1a) (MR 320)
- JVET-AC0104: Block vector difference prediction for IBC blocks (part of AC0113 Test 3.5a) (MR 333)
- JVET-AC0060: IBC BVP candidates clustering and BVD sign derivation for reconstruction-reordered IBC mode (test EE2-3.4, as part of test EE2-3.5a) (MR 335)
- JVET-AC0096: RPR functionality testing (MR 338)
Bug fixes:
- Fix VB handling when SAO is EDGE 135 (MR 304)
- Fix the wrong stride for TIMD (MR 305)
The following adopted aspects were integrated into VTM-11ecm anchor:
- JVET-AB0081: Upscaling filter for display 12-tap luma and 6-tap chroma (MR 341)
The integrated changes do not change VTM-11 anchor performance under CTC.
- JVET-AC0123: MTT correction for class B QP22 (MR 328)
- JVET-AC0096: RPR functionality testing (MR 339)
ECM-8.0 and VTM-11ecm8.0 were tagged on February 24, 2023.
The following bug fixes were integrated into ECM-8.0:
- Fix to prevent it from crashing by calling getIpmInfo() with cs NULL (MR 374)
- Fix OBMC SPS settings based on hash for low-delay config (MR 380)
- Upgrade to C++17 (MR 381)
- Enable BVD prediction and BVP clustering when --EnableTMTools=0, issue #38 (MR 386)
- Fix issue #35 (MR 385)
- Fix IBC-GPM when --EnableTMTools=0 (MR 383)
- Fix propagation of intra modes for IBC blocks, issue #40 (MR 387)
- Pass the correct buffer when CTU is divided by VB (MR 382)
- Use MAX_NUM_REF in regression motion info vector (MR 391)
- Fix GCC 12.2 compiler errors in GCC 12.2, issue #39 (MR 393)
The following fixes were integrated into VTM-11ecm anchor:
- Address GCC12 compiling errors (MR 388)
ECM-8.1 and VTM-11ecm8.1 were tagged on April 18, 2023. The performance of VTM-11ecm8.1 is the same as VTM-11ecm8 under CTC.
CTC Performance
In this section, ECM test results following ECM CTC configuration described in JVET-Y2017 are summarized.
The next tables show ECM-8.0 performance over ECM-7.0 anchor.
All Intra Main 10 | |||||
Over ECM-7.0 | |||||
Y | U | V | EncT | DecT | |
Class A1 | -0.40% | -1.62% | -1.41% | 113% | 108% |
Class A2 | -0.75% | -1.83% | -1.71% | 119% | 110% |
Class B | -0.74% | -1.59% | -1.62% | 117% | 113% |
Class C | -1.22% | -1.79% | -1.72% | 119% | 111% |
Class E | -0.86% | -1.15% | -1.01% | 115% | 109% |
Overall | -0.81% | -1.61% | -1.52% | 117% | 111% |
Class D | -0.88% | -1.65% | -1.70% | 118% | 108% |
Class F | -4.77% | -5.65% | -5.71% | 122% | 113% |
Class TGM | -6.14% | -7.04% | -7.77% | 118% | 109% |
Random Access Main 10 | |||||
Over ECM-7.0 | |||||
Y | U | V | EncT | DecT | |
Class A1 | -0.53% | -1.23% | -1.18% | 109% | 112% |
Class A2 | -1.18% | -1.93% | -2.42% | 109% | 111% |
Class B | -0.72% | -1.49% | -1.53% | 100% | 108% |
Class C | -0.86% | -1.40% | -1.44% | 108% | 108% |
Class E | |||||
Overall | -0.81% | -1.50% | -1.61% | 106% | 109% |
Class D | -0.63% | -1.55% | -1.42% | 109% | 108% |
Class F | -3.87% | -5.14% | -5.27% | 111% | 106% |
Class TGM | -4.71% | -4.90% | -5.03% | 112% | 114% |
Low delay B Main 10 | |||||
Over ECM-7.0 | |||||
Y | U | V | EncT | DecT | |
Class A1 | |||||
Class A2 | |||||
Class B | -0.23% | -0.62% | -0.81% | 104% | 112% |
Class C | -0.32% | -0.78% | -0.71% | 104% | 110% |
Class E | -0.11% | -2.86% | -0.72% | 103% | 108% |
Overall | -0.23% | -1.23% | -0.75% | 104% | 110% |
Class D | -0.22% | -0.70% | -1.28% | 106% | 109% |
Class F | -2.03% | -3.11% | -3.96% | 110% | 110% |
Class TGM | -2.15% | -2.53% | -2.65% | 107% | 115% |
Low delay P Main 10 | |||||
Over ECM-7.0 | |||||
Y | U | V | EncT | DecT | |
Class A1 | |||||
Class A2 | |||||
Class B | -0.12% | -0.66% | -0.90% | 109% | 113% |
Class C | -0.30% | -0.91% | -0.72% | 109% | 111% |
Class E | 0.13% | -1.06% | -0.26% | 107% | 110% |
Overall | -0.12% | -0.84% | -0.68% | 109% | 112% |
Class D | -0.19% | -1.24% | -2.08% | 109% | 108% |
Class F | -2.23% | -3.42% | -3.41% | 111% | 111% |
Class TGM | -2.37% | -2.74% | -3.06% | 111% | 109% |
The below tables show ECM-8.0 performance comparing to VTM-11.0ecm8.0 anchor.
All Intra Main 10 | |||||
Over VTM-11.0ecm8 | |||||
Y | U | V | EncT | DecT | |
Class A1 | -9.03% | -18.78% | -25.18% | 703% | 380% |
Class A2 | -13.53% | -22.78% | -25.89% | 695% | 382% |
Class B | -8.30% | -21.80% | -20.07% | 692% | 421% |
Class C | -9.22% | -14.00% | -14.34% | 677% | 419% |
Class E | -10.49% | -18.86% | -17.70% | 648% | 423% |
Overall | -9.86% | -19.24% | -20.22% | 684% | 407% |
Class D | -7.50% | -11.48% | -11.53% | 672% | 418% |
Class F | -22.22% | -30.58% | -30.24% | 460% | 365% |
Class TGM | -35.58% | -42.92% | -42.31% | 387% | 329% |
Random Access Main 10 | |||||
Over VTM-11.0ecm8 | |||||
Y | U | V | EncT | DecT | |
Class A1 | -19.88% | -22.82% | -29.84% | 687% | 742% |
Class A2 | -22.38% | -29.61% | -32.91% | 632% | 887% |
Class B | -18.26% | -29.12% | -27.08% | 558% | 758% |
Class C | -19.95% | -23.64% | -23.52% | 574% | 757% |
Class E | |||||
Overall | -19.86% | -26.50% | -27.85% | 601% | 779% |
Class D | -20.75% | -25.02% | -25.17% | 594% | 876% |
Class F | -24.95% | -32.44% | -32.74% | 438% | 465% |
Class TGM | -33.27% | -38.66% | -38.88% | 489% | 417% |
Low delay B Main 10 | |||||
Over VTM-11.0ecm8 | |||||
Y | U | V | EncT | DecT | |
Class A1 | |||||
Class A2 | |||||
Class B | -16.36% | -30.93% | -29.37% | 543% | 665% |
Class C | -17.70% | -26.18% | -26.20% | 559% | 671% |
Class E | -14.69% | -24.67% | -23.73% | 545% | 419% |
Overall | -16.39% | -27.78% | -26.90% | 549% | 594% |
Class D | -19.81% | -26.86% | -27.28% | 545% | 768% |
Class F | -22.06% | -31.32% | -30.64% | 448% | 474% |
Class TGM | -30.41% | -36.98% | -36.69% | 458% | 388% |
Low delay P Main 10 | |||||
Over VTM-11.0ecm8 | |||||
Y | U | V | EncT | DecT | |
Class A1 | |||||
Class A2 | |||||
Class B | -15.52% | -30.67% | -28.56% | 501% | 648% |
Class C | -17.16% | -25.51% | -24.94% | 481% | 736% |
Class E | -14.20% | -22.86% | -23.10% | 514% | 481% |
Overall | -15.74% | -26.99% | -25.99% | 497% | 627% |
Class D | -19.95% | -26.98% | -27.49% | 462% | 750% |
Class F | -21.24% | -31.27% | -29.98% | 436% | 436% |
Class TGM | -27.60% | -35.05% | -34.98% | 490% | 361% |
Next tables show ECM-8.1 performance over ECM-8.0 anchor.
All Intra Main 10 | |||||
Over ECM-8.0 | |||||
Y | U | V | EncT | DecT | |
Class A1 | 0.00% | 0.00% | 0.00% | 96% | 98% |
Class A2 | 0.00% | 0.00% | 0.00% | 95% | 98% |
Class B | 0.00% | 0.00% | 0.00% | 96% | 97% |
Class C | 0.00% | 0.00% | 0.00% | 97% | 97% |
Class E | 0.00% | 0.00% | 0.00% | 95% | 95% |
Overall | 0.00% | 0.00% | 0.00% | 96% | 97% |
Class D | 0.00% | 0.00% | 0.00% | 97% | 97% |
Class F | 0.02% | -0.05% | -0.26% | 94% | 94% |
Class TGM | -0.06% | -0.03% | 0.00% | 94% | 95% |
Random Access Main 10 | |||||
Over ECM-8.0 | |||||
Y | U | V | EncT | DecT | |
Class A1 | 0.00% | 0.00% | 0.00% | 97% | 96% |
Class A2 | 0.00% | 0.00% | 0.00% | 97% | 97% |
Class B | 0.00% | 0.00% | 0.00% | 97% | 98% |
Class C | 0.00% | 0.00% | 0.00% | 98% | 98% |
Class E | |||||
Overall | 0.00% | 0.00% | 0.00% | 97% | 97% |
Class D | 0.00% | 0.00% | 0.00% | 97% | 98% |
Class F | 0.01% | -0.36% | 0.06% | 93% | 93% |
Class TGM | -0.01% | 0.02% | 0.03% | 90% | 91% |
Low delay B Main 10 | |||||
Over ECM-8.0 | |||||
Y | U | V | EncT | DecT | |
Class A1 | |||||
Class A2 | |||||
Class B | 0.00% | 0.00% | 0.00% | 97% | 95% |
Class C | 0.00% | 0.00% | 0.00% | 99% | 99% |
Class E | 0.00% | 0.00% | 0.00% | 96% | 99% |
Overall | 0.00% | 0.00% | 0.00% | 97% | 97% |
Class D | 0.00% | 0.00% | 0.00% | 98% | 98% |
Class F | -0.01% | 0.44% | 1.38% | 92% | 92% |
Class TGM | -2.57% | -2.10% | -2.08% | 92% | 93% |
Low delay P Main 10 | |||||
Over ECM-8.0 | |||||
Y | U | V | EncT | DecT | |
Class A1 | |||||
Class A2 | |||||
Class B | 0.00% | 0.00% | 0.00% | 95% | 97% |
Class C | 0.00% | 0.00% | 0.00% | 96% | 98% |
Class E | 0.00% | 0.00% | 0.00% | 94% | 100% |
Overall | 0.00% | 0.00% | 0.00% | 95% | 98% |
Class D | 0.00% | 0.00% | 0.00% | 96% | 99% |
Class F | -0.14% | -0.54% | 0.24% | 92% | 93% |
Class TGM | -1.87% | -1.17% | -1.32% | 93% | 97% |
The Excel files with the complete test results are attached to this report.
ECM memory consumption
ECM memory consumption is provided in ECM encoder log files and is summarized in the table below.
RA | LD | AI | |
class A | 24 GB | 11 GB | |
class B | 13 GB | 12 GB | 9 GB |
class C | 6 GB | 6 GB | 5 GB |
class E | 7 GB | 6 GB | |
class D | 5 GB | 5 GB | 4 GB |
class F | 13 GB | 11 GB | 11 GB |
class TGM | 16 GB | 14 GB | 11 GB |
It is encouraged to care about memory allocation when developing and integrating tools into the ECM. In particular, it is strongly recommended to re-use already allocated memory wherever possible, rather than systematically allocating new memory.
Recommendations
The AHG recommended to:
- Continue to develop ECM software.
- Improve the software documentation.
- Encourage people to report all (potential) bugs that they are finding using GitLab Issues functionality https://vcgit.hhi.fraunhofer.de/ecm/ECM/-/issues.
- Encourage people to submit merge requests fixing identified bugs.