JVET-P0569 AHG17/Non-CE6: High-level syntax for MTS and Implicit Transform Derivations [H. E. Egilmez, V. Seregin, M. Karczewicz (Qualcomm)]
This document proposes modifications for SPS-level signalling of MTS-related flags in order to remove unnecessary dependencies between explicitly signalled and implicitly defined transform selection.
The current design has following issues:
1) Explicit MTS flags sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag can enable implicit MTS derivation even though it is set to be disabled (i.e., VTM-6.0 configuration is set to MTSImplicit = 0). For example, when only inter MTS is enabled and implicit MTS is disabled (i.e., when VTM-6.0 configurations are MTS=2 and MTSImplicit = 0), VTM-6.0 enables implicit transform for intra blocks. In other words, the implicit MTS derivation for intra can depend on inter MTS flag.
2) Flags sps_explicit_mts_intra_enabled_flag, sps_explicit_mts_inter_enabled_flag and sps_mts_enabled_flag can modify the transform derivation for SBT and ISP, which is counter-intuitive. If all these three flags are disabled, only DCT-2 transform can be used in all TUs. On top of this, if only inter MTS flag is enabled, then ISP uses a combination of DCT-2 and DST-7. In other words, transform derivation on ISP (which is an intra coding tool) can depend on inter MTS flag. Similarly, transform derivation for SBT (which is an inter coding tool) can depend on Intra MTS flag.