JVET-Y0101 AHG7/AHG10: Depth motion based fast Multi-Type Tree Splitting [S. Puri, K. Naser, T. Poirier, G. Martin-Cocher (InterDigital)]
A new class of gaming content (DMV) is defined in JVET-Y0041 that includes depth and motion information (motion field or optical flow) retrieved from the frame buffers inside a game engine.
This contribution presents a Depth Motion based Fast Multi-Type Tree (DM_FastMTT) method that uses depth and motion information (e.g. from a game engine) to determine the Multi-Type Tree (MTT) split on the encoder side without expensive RDO. The proposed non-normative method is applied under the low delay/Low Latency Controlled Complexity (LLCC) configurations where the depth and motion information is used to achieve coding gains with negligible complexity increase. The proposed DM_FastMTT method is implemented and tested on VTM-14.0 reference software and vvenc v1.3.0 software.
The overall simulations results on top of VTM-14.0 with MTT disabled and vvenc v1.3.0 lowdelay faster preset are reported as below:
On top of VTM-14.0 with MTT0 (i.e. with MTT splitting disabled)
- LDP: -0.92%/-2.17%/-2.29% with 113% EncT and 95% DecT
On top of vvenc v1.3.0 lowdelay faster preset
- LDP: -1.48%/-1.74%/-1.91% with 122% EncT and 101% DecT
MTT0 loses 8.58% compared to VTM14, with 15% encoder time, this is the anchor of number above (called “test 1” in the contribution) In the contribution, further configurations are tested with higher compression and run time.
Contributors are asked to better refer to VTM14 rather than MTT0 anchor to understand the tradeoff of reduced encoder runtime vs. loss in performance.
The benefit is very specific for this type of gaming sequences which come with depth and motion maps; the algorithm would not work with other sequences.
Does it also give benefit in LB? Yes, according to observation of contributors.
It is pointed out that it may not be suitable for hardware encoders, as more data need to be read. It might be necessary to clarify for which architecture (hardware, software) reduction of complexity is searched for.
Further study of this was recommended.
JVET-Y0101 AHG7/AHG10: Depth motion based fast Multi-Type Tree Splitting [S. Puri, K. Naser, T. Poirier, G. Martin-Cocher (InterDigital)]
See under section 4.10.