JVET-C0001 JVET AHG report: Tool evaluation (AHG1) [M. Karczewicz, E. Alshina]
This document reports the work of the JVET ad hoc group on Tool evaluation (AHG1) between the 2nd JVET meeting at San Diego, USA (20–26 February 2016) and the 3rd Meeting at Geneva, Switzerland (26 May – 1 June 2016).
Joint Exploration Test Model Software (HM-16.6-JEM-2.0) was released 19th of March, 2016. The software can be downloaded at:
https://jvet.hhi.fraunhofer.de/svn/svn_HMJEMSoftware/tags/HM-16.6-JEM-2.0
Algorithms included into JEM2.0 include
- Block structure
- Larger Coding Tree Unit (up to 256x256) and transforms (up to 64x64)
- Quadtree plus binary tree (QTBT) block structure only in EE SW branch
- Intra prediction improvements
- 65 intra prediction directions modified in JEM2.0 compared to JEM1.0
- 4-tap interpolation filter for intra prediction
- Boundary filter applied to other directions in addition to horizontal and vertical ones
- Cross-component linear model (CCLM) prediction
- Position-dependent intra prediction combination (PDPC)
- Adaptive reference sample smoothing
- Inter prediction improvements
- Sub-PU level motion vector prediction
- Locally adaptive motion vector resolution (AMVR)
- 1/16 pel motion vector storage accuracy modified in JEM2.0 compared to JEM1.0
- Overlapped block motion compensation (OBMC)
- Local illumination compensation (LIC)
- Affine motion prediction modified in JEM2.0 compared to JEM1.0
- Pattern matched motion vector derivation
- Bi-directional optical flow (BIO)
- Transform
- Explicit multiple core transform
- Mode dependent non-separable secondary transforms
- Signal dependent transform (SDT) disabled by default
- Adaptive loop filter (ALF)
- Enhanced CABAC design
- Context model selection for transform coefficient levels
- Multi-hypothesis probability estimation
- Initialization for context models
At the 2nd JVET meeting, common test conditions were modified and test sequences were updated. So direct performance comparison between JEM1.0 and JEM2.0 is not especially straightforward.
The table below shows JEM1.0 and JEM2.0 performance compared to the HM if both GOP-size and QP/lambda selection are the same for the test and reference.
JEM coding performance summary in RA test configuration compared to HEVC.
JEM1.0 vs HM16.6 (GOP-size =8 for both) | JEM2.0 vs HM16.9 (GOP-size =16 for both) | |||||
Y | U | V | Y | U | V | |
Class A1 | −22.4% | −21.1% | −24.2% | |||
Class A2 | −28.8% | −29.1% | −22.8% | |||
Class B | −21.3% | −13.2% | −9.2% | −21.8% | −21.5% | −15.8% |
Class C | −20.7% | −14.8% | −18.2% | −20.7% | −19.4% | −21.7% |
Class D | −20.5% | −9.8% | −12.1% | −21.2% | −16.4% | −17.1% |
Overall | −20.8% | −16.7% | −15.4% | −22.9% | −21.5% | −20.1% |
Enc. Time | 6.0 | 5.3 | ||||
Dec. Time | 8.0 | 8.4 | ||||
It was commented that the relationship between QP and lambda has been under study and modification, and noted that further consideration of this issue is under way in the JCT-VC and should be coordinated with JVET. This was taken into account for the table above, but not for the table below.
The net effect of enlarging of GOP size, QP/lambda selection modification and tools up-date in JEM2.0 compared to JEM1.0 is shown in the table below. Test data were provided by the JEM software coordinators.
JEM2.0 (GOP-size =16) vs JEM1.0 (GOP-size =8) performance in RA test.
JEM2.0 vs JEM1.0 (GOP-size is different) | |||
Y | U | V | |
Class A1 | −3.8% | −17.9% | −16.4% |
Class A2 | −4.4% | −17.7% | −15.7% |
Class B | −7.2% | −22.8% | −24.5% |
Class C | −8.2% | −18.7% | −18.1% |
Class D | −7.9% | −20.8% | −20.3% |
Overall | −6.4% | −19.7% | −19.3% |
Enc. Time | 1.0 | ||
Dec. Time | 1.0 | ||
At the 2nd JVET meeting, an Exploration Experiments practice was established. For each new coding tool under consideration special SW branch was created. After implementation of each tool, announcement via the JVET reflector was done. There were 7 exploration experiments on new coding tools. For all of them, input contribution for this meeting were submitted, as listed below.
Authors | ||
EE2.1: Quadtree plus binary tree structure integration with JEM tools | H. Huang, K. Zhang, Y.-W. Huang, S. Lei (MediaTek) | |
EE2.6: Modification of Merge candidate derivation: ATMVP simplification and Merge pruning | S. Lee, W.-J. Chien, L. Zhang, J. Chen, M. Karczewicz (Qualcomm) | |
EE2.5: Improvements on adaptive loop filter | M. Karczewicz, L. Zhang, W.-J. Chien, X. Li (Qualcomm) | |
EE2.3: NSST-PDPC Harmonization | ||
EE2.7: TU-level non-separable secondary transform | X. Zhao, A. Said, V. Seregin, M. Karczewicz, J. Chen, R. Joshi (Qualcomm) | |
EE2.2: Non Square TU Partitioning | K. Rapaka, J. Chen, L. Zhang, W.-J. Chien, M. Karczewicz (Qualcomm) | |
EE2.4: De-quantization and Scaling for Next Generation Containers |
In total, 20 contributions proposing new coding tools for the JEM or improvements of JEM design were submitted in the following categories, as further detailed below:
- Intra (5)
- Inter (6)
- Transform (2)
- Partitioning (2)
- HDR/WCG (1)
- De-blocking (1)
- Other (3)
Authors | Category | ||
Proposed improvements to the Adaptive multiple Core transform | Transform | ||
Predictors Elimination Technique for HEVC | Intra | ||
Simplification of motion compensation filter for affine inter prediction | Inter | ||
Tiles coding improvement for Inter pictures by improved merge list at tile boundaries | Inter | ||
BIO improvement to reduce the encoder and decoder complexities | Inter | ||
Decoupled Luma/Chroma Transform Trees for Intra | F. Urban, T. Poirier, F. Le Léannec (Technicolor) | Partitioning | |
Adaptive Clipping in JEM2.0 | F. Galpin, P. Bordes, F. Le Léannec (Technicolor) | Other | |
Arbitrary reference tier for intra directional modes | Y.-J. Chang, P.-H. Lin, C.-L. Lin, J.-S. Tu, C.-C. Lin (ITRI) | Intra | |
RExt coding tools support on JEM | Other | ||
Generalized bi-prediction for inter coding | Inter | ||
Extended deblocking-filter process for large block boundary | K. Kawamura, Q. Yao, S. Naito (KDDI Corp.) | De-blocking | |
Grouped signalling for transform in QTBT | X. Zhao, V. Seregin, A. Said, M. Karczewicz (Qualcomm) | Partitioning | |
Neighbour based intra most probable modes list derivation | V. Seregin, X. Zhao, A. Said, M. Karczewicz (Qualcomm) | Intra | |
Decoder-side intra mode derivation | Intra | ||
Improved affine motion prediction | F. Zou, J. Chen, M. Karczewicz, X. Li, H.-C. Chuang, W.-J. Chien (Qualcomm) | Inter | |
EE2.7 related: Improved non-separable secondary transform | X. Zhao, A. Said, V. Seregin, M. Karczewicz, J. Chen (Qualcomm) | Transform | |
On Coefficient Scaling | D. B. Sansli, D. Rusanovskyy, J. Sole, A.K. Ramasubramonian, M. Karczewicz (Qualcomm) | HDR/WCG | |
Motion vector coding optimizations | Inter | ||
Direction-dependent scan order with JEM tools | Other | ||
Multiple line-based intra prediction | J. Li (Peking Univ.), B. Li, J. Xu (Microsoft), R. Xiong (Peking Univ.), G.-J. Sullivan (Microsoft) | Intra |
The AHG recommended:
- To review all the related contribution.
- To continue Exploration Experiments practice.
In the next round of tests, it was suggested to also provide results with same bit rate for HEVC/JEM, such that it would be possible to identify whether the compression improvement is visible in terms of subjective quality. This should be done at rate points which are required by applications but not reachable with sufficient quality by HEVC.
It is also noted that the relatively high gain of class A2 may be misleading since some of these sequences are relatively easy to encode (rollercoaster, trafficflow).