JVET-AP0192 [AHG16] Evaluating VTM under HW constraints [K. Naser, F. Le Léannec, F. Galpin, P. de Lagrange, Y. Zhao, E. Alshina (Huawei), Y. Tokumo, T. Ikai (Sharp)]
This contribution studies the VTM encoder under hardware constraints. Specifically, VTM software is tested with constrained number of RDOs, smaller GOP size and reduced number of reference pictures. This setting was recommended in JVET-AO0053.
Using VTM software with RDO extension from JVET-AP0191, the tests were carried out under RA cfg with VTM CTC. The following tests are performed:
Test 1: Impact of small GOP and few references on reduced time environment
- Anchor: VTM-23.9 + RR3 cfg
- Test1-a: Anchor + GOP8 + 2 Ref
- Test1-b: Anchor + GOP8 + 1 Ref
Test 2: Impact of small GOP and few references on reduced time environment and constrained RDOs
- Anchor: VTM-23.9 + RR3 + MaxAllowedIntraRDOs = 10 + MaxAllowedInterRDOs = 10
- Test2-a: Anchor + GOP8 + 2 Ref
- Test2-b: Anchor + GOP8 + 1 Ref
Test 3: Impact of small GOP and few references, reduced run time and constrained RDOs on VTM encoder
- Anchor: VTM-23.9
- Test3-a: VTM-23.9 + RR3 + GOP8 + 2 Ref + MaxAllowedIntraRDOs = 10 + MaxAllowedInterRDOs = 10
- Test3-b: VTM-23.9 + RR3 + GOP8 + 1 Ref + MaxAllowedIntraRDOs = 10 + MaxAllowedInterRDOs = 10
Test 4: Impact of small GOP and few references, reduced run time and more constrained RDOs on VTM encoder
- Anchor: VTM-23.9
- Test4-a: VTM-23.9 + RR3 + GOP8 + 2 Ref + MaxAllowedIntraRDOs = 5 + MaxAllowedInterRDOs = 5
- Test4-b: VTM-23.9 + RR3 + GOP8 + 1 Ref + MaxAllowedIntraRDOs = 5 + MaxAllowedInterRDOs = 5
The following results were obtained:
Test1-a: Anchor: VTM-23.9 + RR3 cfg, Test1-a: Anchor + GOP8 + 2 Ref
Random access Main 10 | |||||
|
| Over VTM-23.9 |
|
| |
Y | U | V | EncT | DecT | |
Class A1 | 6.44% | 13.70% | 14.10% | 99% | 103% |
Class A2 | 7.98% | 15.94% | 13.41% | 98% | 102% |
Class B | 11.15% | 19.50% | 20.02% | 99% | 104% |
Class C | 11.08% | 18.20% | 18.03% | 101% | 110% |
Class E |
|
|
|
| |
Overall | 9.55% | 17.28% | 16.99% | 99% | 105% |
Class D | 10.78% | 18.22% | 17.68% | 100% | 104% |
Class F | 14.23% | 15.79% | 16.16% | 104% | 107% |
|
| Low delay B Main 10 |
|
| |
|
| Over VTM-23.9 |
|
| |
Y | U | V | EncT | DecT | |
Class A1 |
|
|
|
|
|
Class A2 |
|
|
|
| |
Class B | 6.47% | 8.80% | 8.55% | 82% | 97% |
Class C | 10.84% | 14.15% | 13.28% | 83% | 101% |
Class E | 6.04% | 6.30% | 7.37% | 82% | 99% |
Overall | 7.82% | 9.96% | 9.83% | 82% | 99% |
Class D | 7.95% | 10.80% | 9.82% | 81% | 100% |
Class F | 10.03% | 11.23% | 12.88% | 90% | 102% |
Test1-b: Anchor: VTM-23.9 + RR3 cfg, Test1-a: Anchor + GOP8 + 1 Ref
Random access Main 10 | |||||
|
| Over VTM-23.9 |
|
| |
Y | U | V | EncT | DecT | |
Class A1 | 6.68% | 14.21% | 14.72% | 89% | 102% |
Class A2 | 8.01% | 16.22% | 13.60% | 89% | 102% |
Class B | 12.30% | 20.66% | 21.20% | 91% | 104% |
Class C | 12.05% | 19.30% | 19.01% | 93% | 109% |
Class E |
|
|
|
| |
Overall | 10.25% | 18.12% | 17.80% | 91% | 105% |
Class D | 11.38% | 18.72% | 18.39% | 92% | 103% |
Class F | 15.00% | 16.46% | 16.71% | 99% | 108% |
From this test, it can be observed that the impact of setting GOP size to 8 and reducing the number of reference frames to 2 is about 10% BD rate loss. Reducing further the number of reference frames to 1 comes with less than 1% BD rate loss.
Test2-a: Anchor: VTM-23.9 + RR3 + MaxAllowedIntraRDOs = 10 + MaxAllowedInterRDOs = 10, Test: Anchor + GOP8 + 2 Ref
|
| Random access Main 10 |
|
| |
|
| Over Test |
|
| |
Y | U | V | EncT | DecT | |
Class A1 | 6.40% | 14.00% | 13.88% | 98% | 102% |
Class A2 | 7.96% | 15.83% | 13.66% | 97% | 102% |
Class B | 11.19% | 19.40% | 19.87% | 98% | 104% |
Class C | 11.15% | 18.52% | 18.12% | 99% | 108% |
Class E |
|
|
|
| |
Overall | 9.57% | 17.37% | 16.96% | 98% | 104% |
Class D | 10.83% | 18.14% | 17.84% | 99% | 104% |
Class F | 14.41% | 15.77% | 15.94% | 103% | 109% |
|
| Low delay B Main 10 |
|
| |
|
| Over Test |
|
| |
Y | U | V | EncT | DecT | |
Class A1 |
|
|
|
|
|
Class A2 |
|
|
|
| |
Class B | 6.40% | 8.65% | 8.73% | 81% | 101% |
Class C | 10.90% | 14.03% | 13.39% | 81% | 106% |
Class E | 6.16% | 7.14% | 7.74% | 80% | 95% |
Overall | 7.84% | 10.07% | 10.03% | 81% | 101% |
Class D | 7.92% | 10.71% | 10.66% | 80% | 99% |
Class F | 10.17% | 12.96% | 12.87% | 89% | 99% |
Test2-b: Anchor: VTM-23.9 + RR3 + MaxAllowedIntraRDOs = 10 + MaxAllowedInterRDOs = 10, Test: Anchor + GOP8 + 1 Ref
|
| Random access Main 10 |
|
| |
|
| Over Test |
|
| |
Y | U | V | EncT | DecT | |
Class A1 | 6.61% | 14.16% | 14.70% | 88% | 102% |
Class A2 | 7.97% | 16.01% | 13.74% | 87% | 102% |
Class B | 12.37% | 20.56% | 21.22% | 89% | 105% |
Class C | 12.15% | 19.50% | 18.91% | 90% | 108% |
Class E |
|
|
|
| |
Overall | 10.28% | 18.09% | 17.80% | 89% | 104% |
Class D | 11.60% | 18.54% | 17.98% | 90% | 103% |
Class F | 15.21% | 16.51% | 16.73% | 97% | 109% |
From this test, the same conclusion can be drawn. The impact of setting GOP size to 8 and reducing the number of reference frames to 2 is about 10% BD rate loss. Reducing further the number of reference frames to 1 comes with less than 1% BD rate loss.
Test3-a: Anchor: VTM-23.9, Test3-a: VTM-23.9 + RR3 + GOP8 + 2 Ref + MaxAllowedIntraRDOs = 10 + MaxAllowedInterRDOs = 10
|
| Random access Main 10 |
|
| |
|
| Over Test |
|
| |
Y | U | V | EncT | DecT | |
Class A1 | 18.77% | 28.50% | 31.70% | 14% | 105% |
Class A2 | 22.89% | 33.70% | 31.20% | 16% | 105% |
Class B | 22.43% | 35.45% | 37.54% | 15% | 106% |
Class C | 21.37% | 29.88% | 31.29% | 14% | 105% |
Class E |
|
|
|
| |
Overall | 21.51% | 32.22% | 33.44% | 15% | 105% |
Class D | 19.10% | 28.84% | 29.91% | 14% | 98% |
Class F | 29.25% | 31.71% | 33.76% | 22% | 111% |
|
| Low delay B Main 10 |
|
| |
|
| Over Test |
|
| |
Y | U | V | EncT | DecT | |
Class A1 |
|
|
|
|
|
Class A2 |
|
|
|
| |
Class B | 16.99% | 23.25% | 27.15% | 14% | 98% |
Class C | 19.77% | 24.90% | 25.49% | 13% | 95% |
Class E | 18.60% | 21.71% | 22.92% | 18% | 92% |
Overall | 18.32% | 23.41% | 25.54% | 15% | 96% |
Class D | 15.42% | 17.86% | 20.93% | 14% | 90% |
Class F | 27.57% | 32.27% | 33.47% | 21% | 98% |
Test3-b: Anchor: VTM-23.9, Test3-a: VTM-23.9 + RR3 + GOP8 + 1 Ref + MaxAllowedIntraRDOs = 10 + MaxAllowedInterRDOs = 10
|
| Random access Main 10 |
|
| |
|
| Over Test |
|
| |
Y | U | V | EncT | DecT | |
Class A1 | 19.01% | 28.68% | 32.69% | 13% | 105% |
Class A2 | 22.91% | 33.93% | 31.31% | 14% | 105% |
Class B | 23.78% | 36.82% | 39.13% | 14% | 107% |
Class C | 22.49% | 31.00% | 32.22% | 13% | 104% |
Class E |
|
|
|
| |
Overall | 22.31% | 33.06% | 34.43% | 14% | 106% |
Class D | 19.95% | 29.32% | 30.08% | 13% | 98% |
Class F | 30.19% | 32.60% | 34.74% | 21% | 110% |
From this test, the impact of the imposing the hardware constraints on the VTM encoder leads to 21% BD rate loss. With additionally reducing the number of reference frames to 1, less than 1% coding loss is observed.
Test4-a: Anchor: VTM-23.9, Test3-a: VTM-23.9 + RR3 + GOP8 + 2 Ref + MaxAllowedIntraRDOs = 5 + MaxAllowedInterRDOs = 5
|
| Random access Main 10 |
|
| |
|
| Over Test |
|
| |
Y | U | V | EncT | DecT | |
Class A1 | 21.64% | 33.41% | 36.91% | 13% | 107% |
Class A2 | 25.51% | 39.85% | 35.67% | 14% | 106% |
Class B | 25.35% | 42.55% | 45.04% | 14% | 106% |
Class C | 25.19% | 36.83% | 38.97% | 13% | 102% |
Class E |
|
|
|
| |
Overall | 24.60% | 38.66% | 39.92% | 13% | 105% |
Class D | 23.01% | 36.68% | 37.70% | 12% | 98% |
Class F | 37.23% | 43.98% | 47.36% | 19% | 109% |
|
| Low delay B Main 10 |
|
| |
|
| Over Test |
|
| |
Y | U | V | EncT | DecT | |
Class A1 |
|
|
|
|
|
Class A2 |
|
|
|
| |
Class B | 18.49% | 26.93% | 30.90% | 12% | 99% |
Class C | 21.88% | 28.47% | 29.29% | 12% | 97% |
Class E | 20.21% | 25.11% | 24.83% | 17% | 95% |
Overall | 20.05% | 26.99% | 28.85% | 13% | 97% |
Class D | 17.78% | 22.21% | 24.99% | 12% | 91% |
Class F | 33.82% | 39.59% | 42.17% | 19% | 98% |
Test4-b: Anchor: VTM-23.9, Test3-a: VTM-23.9 + RR3 + GOP8 + 1 Ref + MaxAllowedIntraRDOs = 5 + MaxAllowedInterRDOs = 5
|
| Random access Main 10 |
|
| |
|
| Over Test |
|
| |
Y | U | V | EncT | DecT | |
Class A1 | 21.92% | 33.88% | 37.78% | 11% | 107% |
Class A2 | 25.58% | 40.26% | 35.91% | 13% | 104% |
Class B | 26.71% | 44.14% | 46.59% | 13% | 106% |
Class C | 26.31% | 37.83% | 39.75% | 11% | 105% |
Class E |
|
|
|
| |
Overall | 25.42% | 39.63% | 40.87% | 12% | 106% |
Class D | 23.70% | 37.19% | 38.22% | 11% | 99% |
Class F | 38.07% | 44.54% | 48.02% | 18% | 109% |
From this test, the impact of the imposing the hardware constraints with further reducing the number of RDOs to 5 leads to 24% BD rate loss. With additionally reducing the number of reference frames to 1, less than 1% coding loss is observed.
Summary of the results:
GOP | Num Ref | Add cfg | MaxRDOs | Y | EncT | |||||
Anchor | Test | Anchor | Test | Anchor | Test | Anchor | Test | |||
Test1-a | 32 | 8 | Def | 2 | RR3 | RR3 | Def | Def | 9.55% | 99% |
Test1-b | 32 | 8 | Def | 1 | RR3 | RR3 | Def | Def | 10.25% | 91% |
Test2-a | 32 | 8 | Def | 2 | RR3 | RR3 | 10 | 10 | 9.57% | 98% |
Test2-b | 32 | 8 | Def | 1 | RR3 | RR3 | 10 | 10 | 10.28% | 89% |
Test3-a | 32 | 8 | Def | 2 | RR3 | Def | 10 | 21.51% | 15% | |
Test3-b | 32 | 8 | Def | 1 | RR3 | Def | 10 | 22.31% | 14% | |
Test4-a | 32 | 8 | Def | 2 | RR3 | Def | 5 | 24.60% | 13% | |
Test4-b | 32 | 8 | Def | 1 | RR3 | Def | 5 | 25.42% | 12% | |
It was agreed to include the software from JVET-AP0191 in the AHG16 software for further experimentation. Contributions from other companies with experience in hardware encoder design are more than welcome to include more possible approaches of mimicking constraints imposed by hardware encoders (including possibly more sophisticated ones that cause less loss in compression).
It was commented that at this moment it would be premature to define a starting point for restrictions in software of the next generation standard, as it cannot be known what that software would be, and in an initial phase after January it would be most important to establish a stable code base for experimentation.