JVET-T0003 JVET AHG report: Test model software development (AHG3) [F. Bossen, X. Li, K. Sühring]
This report summarizes the activities of the AhG3 on Test model software development that has taken place between the 19th and 20th JVET meetings.
VTM software development
Development was continued on the GitLab server, which allows participants to register accounts and use a distributed development workflow based on git.
The server is located at:
https://vcgit.hhi.fraunhofer.de
The registration and development workflow is documented at:
https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/wikis/VVC-Software-Development-Workflow
The VTM software can be found at
https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/
VTM 9.2 was tagged on July 6, 2020.
VTM 9.3 was tagged on July 6, 2020. Relative to VTM 9.2, only macros of the previous cycle are removed.
VTM 10.0 was tagged Aug. 13, 2020. Changes included were listed in the AHG report.
VTM 10.1 release candidate 1 was tagged on Oct. 6, 2020. It was expected that version 10.1 would be tagged shortly thereafter.
CTC Performance
The following tables show VTM 10.0 BD-Rate performance (measured for PSNR) over HM 16.22:
All Intra Main 10 | |||||
Over HM 16.22 | |||||
Y | U | V | EncT | DecT | |
Class A1 | −29.04% | −32.17% | −34.07% | 1591% | 166% |
Class A2 | −29.29% | −23.92% | −21.06% | 2567% | 172% |
Class B | −21.73% | −26.96% | −30.76% | 2809% | 172% |
Class C | −22.54% | −18.95% | −22.70% | 3951% | 188% |
Class E | −25.76% | −25.91% | −24.46% | 2318% | 165% |
Overall | −25.06% | −25.37% | −26.85% | 2630% | 173% |
Class D | −18.47% | −13.31% | −13.42% | 4447% | 180% |
Class F | −39.33% | −39.73% | −42.22% | 5088% | 174% |
Random access Main 10 | |||||
Over HM 16.22 | |||||
Y | U | V | EncT | DecT | |
Class A1 | −38.76% | −37.00% | −44.06% | 739% | 155% |
Class A2 | −43.20% | −39.82% | −38.41% | 833% | 172% |
Class B | −35.02% | −46.82% | −44.61% | 803% | 153% |
Class C | −29.98% | −30.67% | −32.62% | 1070% | 153% |
Class E | |||||
Overall | −36.06% | −39.15% | −40.06% | 859% | 157% |
Class D | −27.88% | −26.68% | −26.12% | 1175% | 165% |
Class F | −41.52% | −44.62% | −45.73% | 607% | 133% |
Low delay B Main 10 | |||||
Over HM 16.22 | |||||
Y | U | V | EncT | DecT | |
Class A1 | |||||
Class A2 | |||||
Class B | −30.81% | −37.35% | −35.42% | 738% | 142% |
Class C | −29.13% | −22.47% | −22.45% | 890% | 151% |
Class E | −33.36% | −40.20% | −34.18% | 364% | 120% |
Overall | −30.89% | −33.11% | −30.79% | 658% | 139% |
Class D | −26.02% | −16.68% | −15.88% | 917% | 152% |
Class F | −42.80% | −44.45% | −44.71% | 498% | 125% |
Low delay P Main 10 | |||||
Over HM 16.22 | |||||
Y | U | V | EncT | DecT | |
Class A1 | |||||
Class A2 | |||||
Class B | −35.15% | −39.91% | −37.83% | 690% | 158% |
Class C | −30.83% | −22.55% | −22.66% | 818% | 158% |
Class E | −36.05% | −43.41% | −37.33% | 365% | 128% |
Overall | −33.93% | −35.00% | −32.65% | 623% | 150% |
Class D | −27.47% | −15.71% | −14.93% | 856% | 160% |
Class F | −42.31% | −43.56% | −44.09% | 536% | 133% |
The following tables show VTM 10.0 performance compared to VTM 9.0:
All Intra Main 10 | |||||
Over VTM 9.0 | |||||
Y | U | V | EncT | DecT | |
Class A1 | −0.01% | 0.01% | 0.00% | 102% | 101% |
Class A2 | −0.01% | 0.03% | −0.01% | 102% | 99% |
Class B | −0.01% | −0.05% | −0.04% | 101% | 98% |
Class C | −0.02% | 0.01% | 0.01% | 101% | 98% |
Class E | −0.03% | 0.05% | −0.01% | 103% | 99% |
Overall | −0.01% | 0.00% | −0.01% | 101% | 99% |
Class D | −0.06% | −0.03% | −0.08% | 100% | 102% |
Class F | −0.03% | −0.04% | −0.02% | 101% | 96% |
Random access Main 10 | |||||
Over VTM 9.0 | |||||
Y | U | V | EncT | DecT | |
Class A1 | −0.02% | −0.01% | −0.01% | 102% | 101% |
Class A2 | −0.01% | −0.01% | −0.01% | 102% | 99% |
Class B | −0.03% | −0.04% | −0.03% | 100% | 98% |
Class C | −0.04% | −0.04% | −0.04% | 102% | 96% |
Class E | |||||
Overall | −0.03% | −0.03% | −0.03% | 101% | 98% |
Class D | −0.12% | −0.12% | −0.12% | 99% | 98% |
Class F | −0.07% | −0.07% | −0.07% | 102% | 96% |
Low delay B Main 10 | |||||
Over VTM 9.0 | |||||
Y | U | V | EncT | DecT | |
Class A1 | |||||
Class A2 | |||||
Class B | −0.03% | −0.03% | −0.03% | 100% | 98% |
Class C | −0.02% | −0.02% | −0.02% | 99% | 96% |
Class E | −0.46% | −0.47% | −0.42% | 100% | 96% |
Overall | −0.13% | −0.14% | −0.12% | 99% | 97% |
Class D | −0.06% | −0.05% | −0.05% | 99% | 94% |
Class F | −0.20% | −0.20% | −0.20% | 100% | 96% |
Low delay P Main 10 | |||||
Over VTM 9.0 | |||||
Y | U | V | EncT | DecT | |
Class A1 | |||||
Class A2 | |||||
Class B | −0.03% | −0.03% | −0.03% | 98% | 97% |
Class C | −0.02% | −0.01% | −0.01% | 98% | 92% |
Class E | −0.44% | −0.44% | −0.40% | 101% | 95% |
Overall | −0.13% | −0.13% | −0.12% | 99% | 95% |
Class D | −0.05% | −0.04% | −0.04% | 99% | 94% |
Class F | −0.20% | −0.20% | −0.19% | 101% | 98% |
Full results were attached to the AHG report as Excel files.
Issues in VTM 10.0 affecting conformance
The following issues in VTM 10.0 affect conformance:
- A bug related to the combination for deblocking, 4:2:2/4:4:4 chroma formats and ISP (ticket #1353)
- Incorrect output order of pictures (ticket #1396). In this context it should be noted that the Output Picture Log produced by VTM 10.0 doesn’t include layer ids making such issues more difficult to identify and debug.
- Missing HLS features
These issues have been partially resolved in VTM 10.1rc1. In particular the OPL format was changed to include layers ids. Some HLS features are still missing (see sections below) and there are several tickets related to multilayer coding that are still open.
Status of implementation of proposals of previous JVET meetings
Actions of the Q, R, and S meetings that had not yet been marked as resolved were listed in the AHG report.
The bug tracker for VTM and specification text is located at:
https://jvet.hhi.fraunhofer.de/trac/vvc
The bug tracker uses the same accounts as the HM software bug tracker. Users may need to log in again due to the different sub-domain. For spam fighting reasons account registration is only possible at the HM software bug tracker at
https://hevc.hhi.fraunhofer.de/trac/hevc
Please file all issues related to the VVC reference software into the bug tracker. Try to provide all the details, which are necessary to reproduce the issue. Patches for solving issues and improving the software are always appreciated.
The AHG recommended to:
- Continue to develop the VTM reference software
- Improve documentation, especially the software manual
- Resolve any normative issues resulting from the large number of integrations in the most recent development cycle
- Encourage people to test VTM software more extensively outside of common test conditions.
- Encourage people to report all (potential) bugs that they are finding.
- Encourage people to submit bitstreams/test cases that trigger bugs in VTM.
- Encourage people to submit non-normative changes that reduce encoder run time without significantly sacrificing compression performance
- Design and add configuration files to the VTM software for testing of HLS features
- Review VTM-related contributions and determine whether features should be added (or removed) from the software
It was commented that we need people to check the issues that are marked as open to confirm whether any action is really needed and to make the software coordinators aware of the result. The software coordinators also emphasized the need for those who have contributed patches to keep an eye out for email and make sure the issues of its integration have been fully resolved.