JVET-AJ0014 JVET AHG report: NNVC software development (AHG14) [F. Galpin (chair), R. Chang, Y. Li, Y. Li, M. Santamaria, J. N. Shingala, Z. Xie (vice-chairs)]
- Software development
Location
NNVC repository is located at https://vcgit.hhi.fraunhofer.de/jvet-ahg-nnvc/VVCSoftware_VTM
NNVC software is based on VTM-11.0 with enabled MCTF including the update from JVET-V0056, GOP32, enabling deblocking in the RDO, JVET-AH0054 with improved MCTF, JVET-AI0124 for reference picture alignment.
NNVC-10.0 anchor at https://vcgit.hhi.fraunhofer.de/jvet-ahg-nnvc/nnvc-ctc/-/blob/master/Anchor%20performance/NNVC-10.0-VTM_vs_NNVC-10.0.xlsm?ref_type=heads is used for NNVC performance evaluation.
Software changes
NNVC-10.0
Several commits were merged in the NNVC repository. The following changes were integrated:
contribution | MR | topic |
93 SADL | reducemean | |
95 SADL | batchnorm in experimental | |
88, 89, 90 SADL | SADL update | |
245 | encoder stategy, new NNSR | |
239 | vlop lop3 | |
244 | LOP3 adaptive models | |
| LOP3 adaptive code | |
242 | reference images | |
260 | NN intra | |
| VLOP adapt models | |
| VLOP adapt code | |
237 | HOP5 code | |
| HOP5 model | |
JVET-0174 | 91 SADL | 2x2 conv |
62 SADL | padding 0 0 support | |
102 SADL | extended slicing support for JVET-AI0095 |
The following fixes and clean-up were performed:
contribution | MR | topic |
| 104 SADL | better support for experimental layers |
JVET-0155 | 238 | Fix issue 94 |
254 | fix for palette mode | |
| 236 | fix issue 93 |
| 235 | clang build |
| 97 SADL | fix converter |
| 101 SADL | dump issue |
| 100 SADL | slicing 2D |
| 99 SADL | min/max constant |
| 98 SADL | min/max quantization |
| 96 SADL | handle single mutable input |
| 92 SADL | more speedup |
| 107 SADL | prelu extension |
| 258 | fix encoding errors |
Software versions
NNVC-10.0 was tagged August 9th, 2024
NNVC-9.1 was tagged May 28th, 2024
NNVC-9.0 was tagged May 13th, 2024
NNVC-8.0 was tagged February 20th, 2024
NNVC-8.0rc3 was tagged February 20th, 2024.
NNVC-7.1 was tagged November 29th, 2023.
NNVC-7.0 was tagged November 3rd, 2023.
NNVC-6.1 was tagged September 25th, 2023 (fix).
NNVC-6.0 was tagged September 6th, 2023.
NNVC-5.1 was tagged July 19th, 2023.
NNVC-5.0 was tagged May 11th, 2023.
NNVC-3.0 (a.k.a VTM-11.0_nnvc3.0) was tagged December 1st, 2022.
NCS-1.0 (a.k.a NNVC-3.0wip2) was tagged September 4th, 2022 (first release containing the FilterSets, using NNVC 2.0 as a base).
VTM-11.0_nnvc-2.0 was tagged August 4th, 2022 (add deblocking in RDO).
VTM-11.0_nnvc-1.0 was tagged May 6th, 2021 (VTM-11.0 base with MCTF enabled).
Documentation update
Several issues have been opened on the bug tracker related to out-of-date documentation. Several issues were related to the discrepancies arising from the updated version of NNVC (change of behaviour, options etc.). The issues can be especially crucial since it impact the dataset creation for proper training of the models. Several documentation updates have been released. Some issues are still opened and need to be solved. It is encouraged to update the documentation to be compatible with current NNVC software or to specify which version of NNVC to use in order to create a particular dataset.
- CTC performance
See configurations section for naming convention.
Comparison to VTM
NNVC-9.1 VTM vs NNVC-10.0 VTM
AI results are unchanged compared to NNVC-9.1.
Random access Main 10 | ||||||||
BD-rate Over NNVC-9.1 VTM | ||||||||
Y-PSNR | U-PSNR | V-PSNR | Y-MSIM | U-MSIM | V-MSIM | EncT | DecT | |
Class A1 | -0.39% | -0.21% | -0.38% | -0.53% | -0.54% | -0.47% | 103% | 101% |
Class A2 | -0.45% | -0.44% | -0.47% | -0.46% | -0.55% | -0.56% | 105% | 102% |
Class B | -0.62% | -0.66% | -0.61% | -0.59% | -0.67% | -0.65% | 104% | 105% |
Class C | -0.68% | -0.55% | -0.70% | -0.70% | -0.60% | -0.78% | 104% | 108% |
Class E | ||||||||
Overall | -0.56% | -0.50% | -0.56% | -0.59% | -0.60% | -0.63% | 104% | 104% |
Class D | -0.45% | -0.28% | -0.41% | -0.44% | -0.32% | -0.44% | 109% | 115% |
Class F | -0.41% | -0.37% | -0.31% | -0.52% | -0.47% | -0.35% | 102% | 111% |
Class H | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Low delay B Main 10 | ||||||||
BD-rate Over NNVC-9.1 VTM | ||||||||
Y-PSNR | U-PSNR | V-PSNR | Y-MSIM | U-MSIM | V-MSIM | EncT | DecT | |
Class A1 | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Class A2 | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Class B | -0.77% | -2.68% | -1.85% | -1.45% | -3.08% | -1.98% | 99% | 99% |
Class C | -2.43% | -4.19% | -3.71% | -2.90% | -4.82% | -4.99% | 103% | 99% |
Class E | -1.73% | -2.84% | -2.86% | -1.69% | -2.48% | -2.89% | 98% | 99% |
Overall | -1.56% | -3.22% | -2.72% | -1.99% | -3.51% | -3.21% | 100% | 99% |
Class D | -1.79% | -3.55% | -3.06% | -2.23% | -3.13% | -3.13% | 105% | 103% |
Class F | -2.43% | -3.02% | -3.02% | -2.34% | -2.78% | -3.88% | 100% | 99% |
Note: Results from InterDigital, crosschecked by OPPO.
NNVC-10.0 VTM vs NNVC-10.0 anchor
The NNVC-10.0 anchor includes LOP.3 filter and Intra Prediction tools activated.
Random access Main 10 | ||||||||
BD-rate Over NNVC-10.0 VTM | ||||||||
Y-PSNR | U-PSNR | V-PSNR | Y-MSIM | U-MSIM | V-MSIM | EncT | DecT | |
Class A1 | -8.76% | -12.04% | -12.66% | -9.44% | -13.66% | -14.75% | 116% | 3407% |
Class A2 | -7.66% | -12.88% | -8.52% | -8.30% | -13.70% | -8.64% | 115% | 3270% |
Class B | -6.98% | -14.82% | -12.55% | -7.70% | -16.34% | -15.19% | 117% | 3374% |
Class C | -6.60% | -13.78% | -12.09% | -7.62% | -14.74% | -13.32% | 114% | 3097% |
Class E | ||||||||
Overall | -7.37% | -13.60% | -11.64% | -8.15% | -14.85% | -13.29% | 116% | 3284% |
Class D | -6.70% | -12.69% | -11.12% | -6.74% | -13.27% | -12.41% | 111% | 3010% |
Class F | -3.92% | -7.90% | -6.40% | -5.09% | -11.22% | -9.65% | 119% | 1514% |
Class H | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Low delay B Main 10 | ||||||||
BD-rate Over NNVC-10.0 VTM | ||||||||
Y-PSNR | U-PSNR | V-PSNR | Y-MSIM | U-MSIM | V-MSIM | EncT | DecT | |
Class A1 | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Class A2 | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Class B | -5.47% | -10.32% | -8.37% | -6.53% | -9.89% | -13.48% | 101% | 3072% |
Class C | -5.30% | -11.85% | -9.21% | -7.01% | -10.06% | -8.22% | 99% | 2876% |
Class E | -5.97% | -4.97% | -3.90% | -8.07% | -3.45% | -5.30% | 107% | 3184% |
Overall | -5.54% | -9.49% | -7.53% | -7.07% | -8.34% | -9.68% | 102% | 3032% |
Class D | -5.88% | -9.40% | -7.75% | -6.96% | -4.11% | -9.38% | 98% | 2780% |
Class F | -3.15% | -6.55% | -2.54% | -5.59% | -6.65% | -6.84% | 102% | 1570% |
Class H | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Low delay P Main 10 | ||||||||
BD-rate Over NNVC-10.0 VTM | ||||||||
Y-PSNR | U-PSNR | V-PSNR | Y-MSIM | U-MSIM | V-MSIM | EncT | DecT | |
Class A1 | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Class A2 | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Class B | -5.94% | -13.07% | -10.36% | -6.88% | -13.57% | -16.40% | 114% | 3337% |
Class C | -5.65% | -13.98% | -11.26% | -7.20% | -12.90% | -11.64% | 109% | 2981% |
Class E | -6.85% | -6.49% | -4.38% | -9.38% | -5.86% | -8.39% | 118% | 3554% |
Overall | -6.07% | -11.73% | -9.16% | -7.61% | -11.42% | -12.81% | 113% | 3265% |
Class D | -6.27% | -11.84% | -10.03% | -7.37% | -9.11% | -12.28% | 105% | 2760% |
Class F | -3.43% | -7.66% | -3.58% | -5.57% | -9.82% | -10.18% | 116% | 1677% |
Class H | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
All Intra Main 10 | ||||||||
BD-rate Over NNVC-10.0 VTM | ||||||||
Y-PSNR | U-PSNR | V-PSNR | Y-MSIM | U-MSIM | V-MSIM | EncT | DecT | |
Class A1 | -9.23% | -14.19% | -14.86% | -9.37% | -16.83% | -16.35% | 169% | 2508% |
Class A2 | -7.84% | -13.91% | -11.05% | -8.47% | -14.43% | -10.64% | 170% | 2124% |
Class B | -7.69% | -14.19% | -14.43% | -8.12% | -16.15% | -16.37% | 172% | 2161% |
Class C | -7.46% | -13.49% | -13.12% | -8.39% | -16.17% | -15.78% | 156% | 1631% |
Class E | -10.87% | -16.04% | -16.31% | -11.74% | -16.21% | -17.28% | 169% | 2370% |
Overall | -8.45% | -14.30% | -13.96% | -9.05% | -15.99% | -15.44% | 167% | 2107% |
Class D | -7.27% | -11.74% | -12.08% | -7.71% | -14.88% | -15.21% | 151% | 1561% |
Class F | -5.18% | -9.50% | -8.70% | -5.64% | -12.37% | -11.81% | 133% | 1729% |
Note: Results from InterDigital, RA, LDB and AI crosschecked by Nokia.
NNVC-10.0 VTM mode vs NNVC-10.0 HOP.5
The NNVC-10.0 where LOP.3 filter is replaced by HOP.5 and Intra Prediction tools activated.
Random access Main 10 | ||||||||
BD-rate Over NNVC-10.0 VTM | ||||||||
Y-PSNR | U-PSNR | V-PSNR | Y-MSIM | U-MSIM | V-MSIM | EncT | DecT | |
Class A1 | -15.56% | -13.50% | -21.59% | -17.23% | -18.86% | -23.72% | 282% | 131372% |
Class A2 | -15.06% | -20.83% | -26.62% | -14.43% | -20.37% | -23.69% | 264% | 117932% |
Class B | -13.07% | -24.16% | -17.29% | -11.95% | -22.56% | -16.61% | 278% | 115538% |
Class C | -13.92% | -17.30% | -16.90% | -12.52% | -14.51% | -14.24% | 217% | 98690% |
Class E | ||||||||
Overall | -14.19% | -19.53% | -19.91% | -13.65% | -19.24% | -18.82% | 258% | 114132% |
Class D | -14.82% | -17.46% | -17.30% | -11.02% | -12.98% | -11.38% | 211% | 87883% |
Class F | -8.98% | -11.40% | -10.51% | -9.20% | -12.13% | -10.75% | 382% | 51677% |
Class H | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Low delay B Main 10 | ||||||||
BD-rate Over NNVC-10.0 VTM | ||||||||
Y-PSNR | U-PSNR | V-PSNR | Y-MSIM | U-MSIM | V-MSIM | EncT | DecT | |
Class A1 | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Class A2 | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Class B | -9.89% | -12.30% | -7.25% | -9.40% | -9.21% | -5.06% | 253% | 108842% |
Class C | -11.66% | -9.50% | -6.92% | -10.48% | -4.75% | -0.07% | 206% | 92539% |
Class E | -10.61% | -10.60% | -4.64% | -10.49% | -10.87% | -4.12% | 495% | 85823% |
Overall | -10.66% | -10.94% | -6.49% | -10.03% | -8.14% | -3.16% | 279% | 97164% |
Class D | -12.84% | -7.62% | -0.53% | -10.54% | 1.17% | 14.47% | 200% | 93252% |
Class F | -9.13% | -10.02% | -9.67% | -10.27% | -11.13% | -9.55% | 358% | 56653% |
Class H | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Low delay P Main 10 | ||||||||
BD-rate Over NNVC-10.0 VTM | ||||||||
Y-PSNR | U-PSNR | V-PSNR | Y-MSIM | U-MSIM | V-MSIM | EncT | DecT | |
Class A1 | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Class A2 | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Class B | -11.07% | -15.39% | -9.62% | -10.31% | -12.22% | -7.20% | 326% | 117107% |
Class C | -12.58% | -11.57% | -9.68% | -10.96% | -6.13% | -2.93% | 246% | 94481% |
Class E | -11.64% | -10.69% | -4.73% | -11.55% | -10.77% | -3.90% | 651% | 100315% |
Overall | -11.72% | -12.94% | -8.42% | -10.84% | -9.83% | -4.95% | 353% | 104881% |
Class D | -13.93% | -10.90% | -5.70% | -11.51% | -1.74% | 9.66% | 237% | 89160% |
Class F | -9.78% | -12.06% | -11.22% | -10.72% | -13.43% | -12.18% | 431% | 58542% |
Class H | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
All Intra Main 10 | ||||||||
BD-rate Over NNVC-10.0 VTM | ||||||||
Y-PSNR | U-PSNR | V-PSNR | Y-MSIM | U-MSIM | V-MSIM | EncT | DecT | |
Class A1 | -13.52% | -10.72% | -17.21% | -14.99% | -16.58% | -19.47% | 352% | 85675% |
Class A2 | -13.56% | -17.55% | -21.27% | -14.29% | -18.55% | -18.25% | 261% | 65539% |
Class B | -12.12% | -17.04% | -14.38% | -12.02% | -17.10% | -14.82% | 250% | 63431% |
Class C | -13.07% | -14.27% | -16.16% | -12.79% | -13.88% | -15.46% | 197% | 43426% |
Class E | -16.66% | -19.95% | -18.85% | -16.38% | -18.83% | -19.55% | 260% | 74235% |
Overall | -13.56% | -15.94% | -17.14% | -13.79% | -16.83% | -17.10% | 254% | 63277% |
Class D | -12.19% | -12.68% | -15.79% | -11.24% | -12.70% | -14.68% | 179% | 39397% |
Class F | -9.95% | -12.46% | -10.46% | -9.67% | -12.44% | -10.98% | 170% | 53849% |
Note: Results from InterDigital. LDP results crosschecked by Huawei. RA results crosschecked by OPPO.
NNVC-10.0 VTM mode vs NNVC-10.0 VLOP
The NNVC-10.0 where LOP.3 filter is replaced by VLOP2 and Intra Prediction tools activated.
Random access Main 10 | ||||||||
BD-rate Over NNVC-10.0 VTM | ||||||||
Y-PSNR | U-PSNR | V-PSNR | Y-MSIM | U-MSIM | V-MSIM | EncT | DecT | |
Class A1 | -6.97% | -6.66% | -6.09% | -7.45% | -8.56% | -6.57% | 110% | 1601% |
Class A2 | -5.65% | -6.97% | -4.92% | -6.18% | -7.68% | -4.87% | 109% | 1440% |
Class B | -5.26% | -8.82% | -7.25% | -6.11% | -9.99% | -8.40% | 112% | 1575% |
Class C | -4.94% | -7.33% | -6.61% | -5.97% | -8.66% | -7.70% | 111% | 1443% |
Class E | ||||||||
Overall | -5.59% | -7.62% | -6.38% | -6.36% | -8.89% | -7.14% | 111% | 1516% |
Class D | -4.72% | -6.08% | -5.95% | -5.17% | -7.08% | -7.46% | 110% | 1350% |
Class F | -2.81% | -3.99% | -3.19% | -3.75% | -6.09% | -4.87% | 114% | 752% |
Class H | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Low delay B Main 10 | ||||||||
BD-rate Over NNVC-10.0 VTM | ||||||||
Y-PSNR | U-PSNR | V-PSNR | Y-MSIM | U-MSIM | V-MSIM | EncT | DecT | |
Class A1 | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Class A2 | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Class B | -3.86% | -8.94% | -4.84% | -5.28% | -12.05% | -11.43% | 97% | 1513% |
Class C | -3.50% | -7.53% | -5.48% | -5.31% | -10.72% | -8.67% | 96% | 1358% |
Class E | -4.01% | -3.16% | -3.04% | -6.29% | -7.91% | -7.53% | 98% | 1556% |
Overall | -3.78% | -7.03% | -4.60% | -5.54% | -10.57% | -9.54% | 97% | 1470% |
Class D | -3.96% | -5.37% | -3.23% | -5.21% | -9.97% | -9.69% | 95% | 1282% |
Class F | -1.76% | -2.26% | -1.66% | -3.46% | -5.48% | -5.78% | 95% | 737% |
Class H | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Low delay P Main 10 | ||||||||
BD-rate Over NNVC-10.0 VTM | ||||||||
Y-PSNR | U-PSNR | V-PSNR | Y-MSIM | U-MSIM | V-MSIM | EncT | DecT | |
Class A1 | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Class A2 | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Class B | -4.07% | -10.17% | -5.29% | -5.29% | -13.48% | -12.44% | 110% | 1636% |
Class C | -3.67% | -8.38% | -5.87% | -4.96% | -12.00% | -9.44% | 108% | 1406% |
Class E | -4.48% | -5.26% | -3.00% | -6.95% | -10.07% | -9.27% | 110% | 1705% |
Overall | -4.04% | -8.35% | -4.91% | -5.59% | -12.13% | -10.65% | 109% | 1572% |
Class D | -4.13% | -6.93% | -3.87% | -5.60% | -11.95% | -11.55% | 105% | 1294% |
Class F | -1.83% | -2.93% | -1.57% | -3.34% | -6.80% | -6.51% | 110% | 800% |
Class H | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
All Intra Main 10 | ||||||||
BD-rate Over NNVC-10.0 VTM | ||||||||
Y-PSNR | U-PSNR | V-PSNR | Y-MSIM | U-MSIM | V-MSIM | EncT | DecT | |
Class A1 | -8.07% | -9.46% | -8.69% | -8.22% | -11.63% | -9.14% | 169% | 1441% |
Class A2 | -6.70% | -8.86% | -7.44% | -7.13% | -9.22% | -7.00% | 173% | 1324% |
Class B | -6.59% | -9.45% | -9.06% | -7.04% | -10.75% | -10.05% | 172% | 1316% |
Class C | -6.09% | -8.46% | -8.03% | -7.10% | -10.25% | -9.57% | 160% | 1104% |
Class E | -9.15% | -10.86% | -10.71% | -9.89% | -10.79% | -10.46% | 168% | 1380% |
Overall | -7.17% | -9.37% | -8.77% | -7.74% | -10.54% | -9.35% | 168% | 1296% |
Class D | -5.87% | -7.23% | -7.48% | -6.49% | -9.27% | -10.01% | 148% | 1017% |
Class F | -4.07% | -5.75% | -5.46% | -4.54% | -7.20% | -7.08% | 132% | 972% |
Note: Results from InterDigital, RA, LDB and AI crosschecked by Nokia.
NNVC-10.0 VTM vs NNVC-10.0 LOP3 adaptive content
Random access Main 10 | ||||||||
BD-rate Over NNVC-v10rc-VTM | ||||||||
Y-PSNR | U-PSNR | V-PSNR | Y-MSIM | U-MSIM | V-MSIM | EncT | DecT | |
Class A1 | -10.82% | -14.26% | -18.88% | -12.93% | -17.19% | -20.31% | 220% | 3324% |
Class A2 | -8.16% | -15.70% | -12.03% | -8.23% | -17.01% | -13.38% | 226% | 3189% |
Class B | -7.71% | -20.78% | -17.16% | -6.25% | -22.05% | -19.13% | 221% | 3506% |
Class C | -6.78% | -14.41% | -13.34% | -7.34% | -16.01% | -14.91% | 216% | 3402% |
Class E | ||||||||
Overall | -8.17% | -16.76% | -15.46% | -8.27% | -18.46% | -17.09% | 221% | 3376% |
Class D | -6.66% | -12.56% | -11.15% | -6.59% | -13.08% | -12.35% | 212% | 3348% |
Class F | -3.94% | -8.78% | -7.34% | -4.64% | -11.61% | -10.46% | 230% | 2054% |
Note: Results from Nokia.
NNVC-10.0 VTM vs NNVC-10.0 VLOP adaptive content
Random access Main 10 | ||||||||
BD-rate Over NNVC-v10rc-VTM | ||||||||
Y-PSNR | U-PSNR | V-PSNR | Y-MSIM | U-MSIM | V-MSIM | EncT | DecT | |
Class A1 | -7.99% | -5.31% | -7.27% | -8.98% | -5.64% | -7.14% | 225% | 2677% |
Class A2 | -5.85% | -5.71% | -4.42% | -5.43% | -5.91% | -3.28% | 224% | 2426% |
Class B | -6.41% | -11.19% | -7.93% | -4.89% | -10.52% | -7.26% | 227% | 2857% |
Class C | -5.29% | -6.83% | -6.15% | -5.64% | -7.16% | -5.21% | 225% | 3724% |
Class E | ||||||||
Overall | -6.31% | -7.75% | -6.62% | -6.02% | -7.73% | -5.89% | 226% | 2929% |
Class D | -5.32% | -6.53% | -6.40% | -5.03% | -5.97% | -5.45% | 223% | 5908% |
Class F | -3.12% | -4.79% | -4.13% | -3.23% | -5.38% | -3.52% | 235% | 3066% |
Note: Results from Nokia.
NNVC-10.0 VTM vs NNVC-10.0 NNSR
Random access Main 10 | ||||||||
BD-rate Over NNVC-10 | ||||||||
Y-PSNR | U-PSNR | V-PSNR | Y-MSIM | U-MSIM | V-MSIM | EncT | DecT | |
Class A1 | -3.04% | 7.70% | 6.15% | -6.05% | -0.72% | 0.52% | 112% | #DIV/0! |
Class A2 | -0.99% | 7.89% | 6.82% | -4.29% | 8.14% | 7.05% | 90% | #DIV/0! |
Class B | 0.00% | 0.00% | 0.00% | 0.00% | 0.00% | 0.00% | 100% | 100% |
Class C | 0.00% | 0.00% | 0.00% | 0.00% | 0.00% | 0.00% | 100% | 100% |
Class E | ||||||||
Overall | -0.81% | 3.12% | 2.59% | -2.07% | 1.48% | 1.52% | 100% | #DIV/0! |
Class D | 0.00% | 0.00% | 0.00% | 0.00% | 0.00% | 0.00% | 100% | 100% |
Class F | 0.00% | 0.00% | 0.00% | 0.00% | 0.00% | 0.00% | 100% | 100% |
All Intra Main 10 | ||||||||
BD-rate Over NNVC-10 | ||||||||
Y-PSNR | U-PSNR | V-PSNR | Y-MSIM | U-MSIM | V-MSIM | EncT | DecT | |
Class A1 | -1.65% | 6.88% | 7.68% | -4.62% | 4.87% | 4.84% | 125% | #DIV/0! |
Class A2 | -3.21% | 10.29% | 10.64% | -9.16% | 8.53% | 8.64% | 98% | #DIV/0! |
Class B | 0.00% | 0.00% | 0.00% | 0.00% | 0.00% | 0.00% | 100% | 100% |
Class C | 0.00% | 0.00% | 0.00% | 0.00% | 0.00% | 0.00% | 100% | 100% |
Class E | 0.00% | 0.00% | 0.00% | 0.00% | 0.00% | 0.00% | 100% | 100% |
Overall | -0.81% | 2.86% | 3.05% | -2.30% | 2.23% | 2.25% | 103% | #DIV/0! |
Class D | 0.00% | 0.00% | 0.00% | 0.00% | 0.00% | 0.00% | 100% | 100% |
Class F | 0.00% | 0.00% | 0.00% | 0.00% | 0.00% | 0.00% | 100% | 100% |
Note: Results from vivo.
Comparison to previous NNVC anchor
NNVC-9.1 anchor vs NNVC-10.0 anchor
The results reflect NN intra, VTM base and spee-up progress between NNVC 9.1 (as anchor) and NNVC 10.0 (as test).
Random access Main 10 | ||||||||
BD-rate Over NNVC-9.1 NNVC | ||||||||
Y-PSNR | U-PSNR | V-PSNR | Y-MSIM | U-MSIM | V-MSIM | EncT | DecT | |
Class A1 | -0.32% | -0.49% | -0.52% | -0.49% | -0.96% | -0.82% | 98% | 41% |
Class A2 | -0.55% | -1.19% | -0.98% | -0.61% | -1.41% | -1.36% | 100% | 42% |
Class B | -0.71% | -1.45% | -1.11% | -0.78% | -1.83% | -1.58% | 99% | 43% |
Class C | -0.76% | -0.90% | -0.86% | -0.80% | -1.37% | -1.28% | 99% | 43% |
Class E | ||||||||
Overall | -0.61% | -1.06% | -0.90% | -0.69% | -1.45% | -1.30% | 99% | 42% |
Class D | -0.45% | -1.29% | -0.40% | -0.37% | -1.86% | -0.95% | 100% | 46% |
Class F | -0.51% | -0.79% | -0.64% | -0.59% | -1.06% | -0.84% | 92% | 44% |
Class H | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Low delay B Main 10 | ||||||||
BD-rate Over NNVC-9.1 NNVC | ||||||||
Y-PSNR | U-PSNR | V-PSNR | Y-MSIM | U-MSIM | V-MSIM | EncT | DecT | |
Class A1 | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Class A2 | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Class B | -0.71% | -3.11% | -2.44% | -1.43% | -3.55% | -3.01% | 87% | 38% |
Class C | -2.39% | -4.16% | -3.89% | -2.99% | -4.70% | -4.29% | 90% | 41% |
Class E | -2.06% | -3.52% | -3.44% | -1.92% | -3.83% | -3.67% | 80% | 38% |
Overall | -1.61% | -3.56% | -3.17% | -2.07% | -4.00% | -3.60% | 86% | 39% |
Class D | -1.60% | -2.69% | -2.93% | -2.06% | -2.03% | -3.51% | 92% | 42% |
Class F | -2.54% | -3.42% | -3.32% | -2.83% | -3.18% | -3.32% | 84% | 41% |
All Intra Main 10 | ||||||||
BD-rate Over NNVC-9.1 NNVC | ||||||||
Y-PSNR | U-PSNR | V-PSNR | Y-MSIM | U-MSIM | V-MSIM | EncT | DecT | |
Class A1 | 0.00% | -0.05% | 0.06% | 0.07% | -0.20% | 0.01% | 98% | 45% |
Class A2 | -0.33% | -0.44% | -0.47% | -0.24% | -0.46% | -0.46% | 103% | 45% |
Class B | -0.09% | -0.16% | -0.12% | -0.10% | -0.30% | -0.40% | 103% | 48% |
Class C | -0.02% | -0.11% | -0.01% | -0.06% | -0.10% | -0.06% | 98% | 47% |
Class E | 0.09% | -0.08% | -0.15% | 0.11% | -0.01% | -0.40% | 101% | 47% |
Overall | -0.07% | -0.16% | -0.13% | -0.05% | -0.22% | -0.26% | 101% | 47% |
Class D | -0.02% | -0.08% | 0.16% | -0.12% | -0.29% | 0.28% | 99% | 51% |
Class F | 0.02% | -0.02% | -0.07% | -0.05% | -0.32% | -0.16% | 92% | 46% |
Note: Results from InterDigital, crosschecked by OPPO.
Comparison to NNVC-10.0 anchor
NNVC-10.0 vs NNVC-10.0 HOP.5
The results below reflect LOP/HOP performance differences (LOP set as anchor, HOP as test).
Random access Main 10 | ||||||||
BD-rate Over NNVC-10.0 VTM | ||||||||
Y-PSNR | U-PSNR | V-PSNR | Y-MSIM | U-MSIM | V-MSIM | EncT | DecT | |
Class A1 | -7.43% | -1.96% | -10.34% | -8.59% | -6.50% | -10.77% | 243% | 3856% |
Class A2 | -8.00% | -9.13% | -19.74% | -6.65% | -7.81% | -16.56% | 231% | 3606% |
Class B | -6.58% | -11.32% | -5.55% | -4.59% | -7.92% | -1.70% | 237% | 3424% |
Class C | -7.86% | -4.07% | -5.52% | -5.29% | 0.32% | -1.03% | 190% | 3187% |
Class E | ||||||||
Overall | -7.38% | -7.08% | -9.34% | -5.99% | -5.42% | -6.31% | 223% | 3476% |
Class D | -8.72% | -5.49% | -7.15% | -4.58% | 0.39% | 1.04% | 189% | 2920% |
Class F | -5.36% | -3.86% | -4.41% | -4.37% | -1.07% | -1.21% | 321% | 3413% |
Class H | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Low delay B Main 10 | ||||||||
BD-rate Over NNVC-10.0 VTM | ||||||||
Y-PSNR | U-PSNR | V-PSNR | Y-MSIM | U-MSIM | V-MSIM | EncT | DecT | |
Class A1 | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Class A2 | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Class B | -4.69% | -2.15% | 1.14% | -3.09% | 1.22% | 9.13% | 250% | 3543% |
Class C | -6.74% | 2.72% | 2.23% | -3.75% | 6.21% | 8.74% | 209% | 3218% |
Class E | -4.94% | -5.69% | -0.59% | -2.58% | -7.40% | 1.20% | 464% | 2696% |
Overall | -5.44% | -1.41% | 1.07% | -3.18% | 0.73% | 7.02% | 275% | 3204% |
Class D | -7.41% | 1.65% | 7.23% | -3.84% | 5.80% | 24.56% | 205% | 3354% |
Class F | -6.22% | -3.74% | -7.15% | -4.99% | -4.83% | -2.85% | 349% | 3608% |
Class H | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Low delay P Main 10 | ||||||||
BD-rate Over NNVC-10.0 VTM | ||||||||
Y-PSNR | U-PSNR | V-PSNR | Y-MSIM | U-MSIM | V-MSIM | EncT | DecT | |
Class A1 | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Class A2 | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
Class B | -5.48% | -2.65% | 0.59% | -3.70% | 2.19% | 10.40% | 287% | 3509% |
Class C | -7.36% | 2.81% | 1.48% | -4.07% | 7.86% | 9.43% | 226% | 3169% |
Class E | -5.14% | -4.25% | -0.14% | -2.34% | -4.92% | 4.61% | 553% | 2823% |
Overall | -6.02% | -1.23% | 0.71% | -3.49% | 2.30% | 8.63% | 312% | 3212% |
Class D | -8.17% | 1.02% | 4.22% | -4.49% | 8.52% | 21.84% | 225% | 3231% |
Class F | -6.64% | -4.81% | -7.81% | -5.49% | -4.13% | -2.19% | 373% | 3491% |
Class H | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! |
All Intra Main 10 | ||||||||
BD-rate Over NNVC-10.0 VTM | ||||||||
Y-PSNR | U-PSNR | V-PSNR | Y-MSIM | U-MSIM | V-MSIM | EncT | DecT | |
Class A1 | -4.70% | 3.96% | -2.73% | -6.15% | 0.26% | -3.84% | 209% | 3417% |
Class A2 | -6.23% | -3.79% | -11.29% | -6.31% | -4.34% | -8.35% | 153% | 3086% |
Class B | -4.83% | -3.19% | 0.09% | -4.22% | -1.17% | 1.97% | 145% | 2935% |
Class C | -6.12% | -0.85% | -3.62% | -4.79% | 2.84% | 0.26% | 126% | 2662% |
Class E | -6.53% | -4.70% | -3.05% | -5.17% | -3.17% | -2.79% | 154% | 3132% |
Overall | -5.61% | -1.83% | -3.62% | -5.17% | -0.90% | -1.89% | 152% | 3003% |
Class D | -5.33% | -1.09% | -4.45% | -3.83% | 2.70% | 0.24% | 119% | 2524% |
Class F | -5.13% | -3.35% | -1.92% | -4.26% | -0.11% | 1.08% | 128% | 3114% |
Note: Results from InterDigital.
NNVC-10.0 vs NNVC-10.0 VLOP2
The results below reflect LOP/VLOP performance differences (LOP set as anchor, VLOP as test).
Random access Main 10 | |||||||||
BD-rate Over NNVC-10.0 VTM | |||||||||
Y-PSNR | U-PSNR | V-PSNR | Y-MSIM | U-MSIM | V-MSIM | EncT | DecT | ||
Class A1 | 1.98% | 6.41% | 7.57% | 2.19% | 6.44% | 9.83% | 95% | 47% | |
Class A2 | 2.18% | 7.22% | 4.03% | 2.29% | 7.57% | 4.21% | 95% | 44% | |
Class B | 1.85% | 7.24% | 6.16% | 1.72% | 8.00% | 8.26% | 96% | 47% | |
Class C | 1.79% | 7.64% | 6.33% | 1.77% | 7.53% | 6.75% | 97% | 47% | |
Class E | |||||||||
Overall | 1.93% | 7.18% | 6.06% | 1.94% | 7.47% | 7.36% | 96% | 46% | |
Class D | 2.13% | 7.69% | 5.97% | 1.68% | 7.41% | 5.85% | 99% | 45% | |
Class F | 1.17% | 4.48% | 3.48% | 1.43% | 6.27% | 5.61% | 96% | 50% | |
Class H | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! | |
Low delay B Main 10 | |||||||||
BD-rate Over NNVC-10.0 VTM | |||||||||
Y-PSNR | U-PSNR | V-PSNR | Y-MSIM | U-MSIM | V-MSIM | EncT | DecT | ||
Class A1 | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! | |
Class A2 | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! | |
Class B | 1.72% | 1.72% | 3.91% | 1.34% | -2.11% | 2.45% | 96% | 49% | |
Class C | 1.90% | 4.99% | 4.23% | 1.85% | -0.70% | -0.10% | 98% | 47% | |
Class E | 2.07% | 2.01% | 0.92% | 1.86% | -4.21% | -2.14% | 91% | 49% | |
Overall | 1.87% | 2.88% | 3.27% | 1.64% | -2.17% | 0.45% | 95% | 48% | |
Class D | 2.04% | 4.52% | 4.98% | 1.89% | -5.01% | -0.08% | 97% | 46% | |
Class F | 1.45% | 4.73% | 1.00% | 2.30% | 1.40% | 1.23% | 93% | 47% | |
Class H | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! | |
Low delay P Main 10 | |||||||||
BD-rate Over NNVC-10.0 VTM | |||||||||
Y-PSNR | U-PSNR | V-PSNR | Y-MSIM | U-MSIM | V-MSIM | EncT | DecT | ||
Class A1 | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! | |
Class A2 | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! | |
Class B | 1.99% | 3.54% | 5.66% | 1.72% | 0.56% | 4.90% | 97% | 49% | |
Class C | 2.11% | 6.55% | 6.22% | 2.43% | 1.01% | 2.93% | 99% | 47% | |
Class E | 2.54% | 1.46% | 1.39% | 2.62% | -3.94% | -0.72% | 94% | 48% | |
Overall | 2.17% | 4.02% | 4.78% | 2.18% | -0.41% | 2.84% | 97% | 48% | |
Class D | 2.29% | 6.09% | 6.87% | 1.91% | -1.72% | 1.14% | 100% | 47% | |
Class F | 1.67% | 5.32% | 2.11% | 2.39% | 3.57% | 4.09% | 95% | 48% | |
Class H | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #VALUE! | #DIV/0! | #DIV/0! | |
All Intra Main 10 | |||||||||
BD-rate Over NNVC-10.0 VTM | |||||||||
Y-PSNR | U-PSNR | V-PSNR | Y-MSIM | U-MSIM | V-MSIM | EncT | DecT | ||
Class A1 | 1.29% | 5.77% | 7.37% | 1.25% | 6.79% | 8.79% | 100% | 57% | |
Class A2 | 1.24% | 6.18% | 4.19% | 1.44% | 6.46% | 4.16% | 102% | 62% | |
Class B | 1.21% | 5.65% | 6.37% | 1.15% | 6.71% | 7.71% | 100% | 61% | |
Class C | 1.49% | 6.05% | 6.05% | 1.40% | 7.56% | 7.69% | 102% | 68% | |
Class E | 1.93% | 6.30% | 6.92% | 2.04% | 6.76% | 8.47% | 100% | 58% | |
Overall | 1.41% | 5.95% | 6.19% | 1.42% | 6.88% | 7.42% | 101% | 62% | |
Class D | 1.52% | 5.24% | 5.48% | 1.33% | 6.80% | 6.49% | 98% | 65% | |
Class F | 1.19% | 4.37% | 3.69% | 1.15% | 6.33% | 5.59% | 99% | 56% | |
Note: Results from InterDigital.
- Contributions
Five contributions related to AhG14 were noted.
[AHG11] [AHG14] Teleconference on NNVC | E. Alshina, F. Galpin | |
AHG14: The extension of SADL library | N. Fu, W. Bao, Z. Chen (Wuhan Univ.) | |
AhG14: SADL implementation improvements for sparse models | F. Urban, F. Galpin, T. Dumas (InterDigital) | |
AhG14: SADL update | ||
AHG14: Update of Guidelines for NNVC software development | F. Galpin, E. François (InterDigital) |
Configurations
The following configurations were used to generate the different NNVC results.
The column “tested” is read as follow:
- Y: the configuration has been tested using the new NNVC software
- P: the results are the ones from previous NNVC software basis
- N: not tested.
The column “xcheck” is read as follow:
- Y: the test has been cross-checked
- P: no cross-checked performed but results are consistent with previous version on NNVC
- N: no cross-check available
Name | Tools | Configuration | Tested | Xcheck |
NNVC VTM mode | none | encoder_xxx_vtm.cfg | Y | |
NNVC Anchor/EE1 | Intra Pred + LOP.3 | encoder_xxx_nnvc.cfg | Y | |
NNVC. HOP | Intra Pred + HOP.5 | encoder_xxx_nnvc.cfg + nn-based/HOP4.cfg | Y | |
HOP only | HOP | encoder_xxx_vtm.cfg + nn-based/HOP4.cfg | ||
NN Intra lower complexity | nn-based/intra2.cfg | |||
Super-resolution | nn-based/nnsr.cfg + nn-based/nnsr_classAx.cfg | |||
Pf | Adaptive post-filters | nn-based/nnpf/nnpf_xxx.cfg | ||
ALOP | Intra pred+adaptive LOP | encoder_xxx_vtm.cfg +nn-based/intra2.cfg+xxx |
Deprecated options:
set0 | Loop filter set #0 | encoder_xxx_vtm.cfg + nn-based/NnlfOption_1.cfg | P | |
set1 | Loop filter set #1 | encoder_xxx_vtm.cfg + nn-based/NnlfOption_2.cfg | P | |
set0+rdo | Loop filter set #0 + Rdo | nn-based/NnlfOption_1.cfg + --EncNnlfOpt=1 | P | |
set1+rdo+intra+temporal filter | Loop filter set #1 + Temporal filter | nn-based/NnlfOption_2.cfg + --NnlfSet1Multiframe=1 | P |
- Recommendations
The AHG recommended to:
- Continue to develop NNVC software.
- Improve the software documentation.
- Encourage people to report all (potential) bugs that they are finding using GitLab Issues functionality https://vcgit.hhi.fraunhofer.de/jvet-ahg-nnvc/VVCSoftware_VTM/-/issues
Encourage people to submit merge requests fixing identified bugs.