JVET-D0010 Exploration Experiments on Coding Tools Report [E. Alshina, J. Boyce, X. Xiu, Y. J. Chang, V. Seregin, H. Kim]
Summary of Exploration Experiments.
# | Tests and sub-tests | Document | Y-BD-rate (Enc/DecTime) | Cross-check |
1 | Secondary transform (HyGT) and combination of PDPC and NSST
| (Qualcomm) | AI: −1.1% (ET 0.99, DT 1.00) RA: −0.6% (ET 1.00, DT 1.00) | Samsung Sony |
| JVET-D0168 (Sharp) | Not needed | ||
2 |
| JVET-D0065 (Orange) | AI: −0.3% (ET 1.01, DT 0.99) RA: −0.1% (ET 1.01, DT 0.96) | JVET-D0066 Technicolor JVET-D0096 Samsung Sony |
| (Qualcomm) | AI: −0.2% (ET 1.04, DT 0.97) RA: −0.1% (ET 1.0, DT 1.0) | ||
3 | JVET-D0102 (InterDigital) | RA: −0.8% (ET1.70, DT1.02) LD: −0.9% (ET2.00, DT1.00) | Sharp Samsung | |
4 | JVET-D0121 (Qualcomm) | RA: −0.3% (ET 1.06, DT 1.00) LD: −0.3% (ET 1.11, DT 1.00) LDP: −0.3% (ET 1.10, DT 1.00) | JVET-D0057 Samsung | |
5 |
| JVET-D0062 (Ericsson) | RA: −0.4% (ET 1.20, DT 1.0) LD: −0.1% (ET 1.28, DT 1.0) LDP: −0.1% (ET 1.00, DT 1.0) | Samsung Qualcomm |
6 | Extended intra prediction reference
| (ITRI) | Adaptive selection between 4 tiers AI: −0.9% (ET 2.03, DT 1.03) RA: −0.6% (ET 1.25, DT 0.99) | InterDigital Sony Sharp Qualcomm |
| JVET-D0149 (Microsoft) | Prediction from 4 tiers AI: −0.3% (ET 2.6, DT 1.0) RA: −0.0% (ET 1.3, DT 1.0) | ||
7 | Adaptive clipping
| JVET-D0033 (Technicolor) | AI: −0.2% (ET 0.97, DT 1.05) RA: −0.4% (ET 0.99, DT 1.01) LD: −0.3% (ET 1.00, DT 0.95) LDP: −0.4% (ET 0.99, DT 1.00) | Orange Samsung |
8 | Decoder side intra mode derivation
| JVET-D0097 (InterDigital) | AI: −0.8% (ET 1.64, DT 1.21) RA: −0.5% (ET 1.15, DT 1.01) | Samsung Huawei Sharp ITRI |
9 | Adaptive Scaling for Extended Colour Volume Material (**) | Qualcomm Sharp |
Comments:
(*) For items shown in blue font, only partial results were available (Oct. 12), yet to be updated.
(**) Since EE9 was based on JEM3.1 (not JEM3.0) and test materials for EE9 need to be defined by AhG7, only preliminary results were available for EE9 and it was decided not to request cross-check for those tests at this stage, considering the EE9 results for information only.
EE1: Secondary Transforms (5)
Primary (4)
From summary report JVET-D0010:
This experiment is targeting exploration following modification of secondary transforms proposed in 3 contributions:
- New secondary transform kernel, so-called Hypercubic Givens transform (HyGT) as a replacement of the current JEM secondary transforms. This includes:
- New 44 secondary transforms;
- New 88 secondary transforms;
- Secondary transform is not applied to LM chroma mode and secondary transform index is not signalled.
- 2 or 4 secondary transform candidates are used
- Decoupling PDPC and NSST by removing the bit-stream restriction currently precluding enabling NSST and PDPC at the same time.
Questions recommended to be answered during EE tests:
[Q]: What is the performance effect of 44 HyGT vs existing JEM secondary transforms?
[A]: 0.0% (AI) / 0.0% (RA) (EE1-T1), similar performance for Chroma.
[Q]: What is the memory usage for storage of 44 HyGT compared to existing JEM secondary transforms?
[A]: Existing JEM secondary transforms (all are 44) require 10368 bytes. HyGT 44 implemented using 8264 bytes. Please note JEM 2nd transforms are implemented as matrix multiplication, HyGT is series of permutations and Given’s rotations (so memory size is smaller).
[Q]: What is the performance effect from replacement of 44 HyGT by 88 HyGT in large blocks?
[A]: 1.1% (AI) / 0.6% (RA) (EE1-T1), similar gain is observed for Chroma.
[Q]:What is the memory usage for storage of 88 HyGT compared 44 HyGT?
[A]: HyGT 88 implemented using 85680 bytes
Conclusion from EE1 report: replacement of existing secondary transform design by combination of 44 HyGT for small blocks and 88 HyGT for larger blocks provides 1.1% (AI ) and 0.6% (RA) gain w/o noticeable run-time increment. Memory usage for secondary transform matrices storage increases from 10368 bytes (in JEM3.0) up to 93944 bytes.
From JVET discussion:
Can HyGT also be implemented as matrix multiply, or can current transform be implemented as butterfly or similar?
The replacement of the existing 4x4 transform is not interesting, as the advantage may be implementation-specific, and no compression gain is achieved. The 8x8 case gives interesting gain, without increase of encoder/decoder runtime.
Using the same transform for 4x4 and 8x8 is more consistent in the design and shares code.
It was also commented that reducing memory usage should be further studied.
Decision: Adopt D0120 for both 4x4 and 8x8 secondary transform.