JVET-R0083 AHG14: Residual coding constraints for transform skip blocks [A. Nalci, H.E. Egilmez, M. Coban, V. Seregin, M. Karczewicz (Qualcomm)]
In 17th JVET meeting, the slice level flag “slice_ts_residual_coding_disabled_flag” was adopted into VVC as part of JVET-Q0089. This flag can bypass transform skip residual coding (TSRC) and enables the use of regular residual coding (RRC) for TS blocks. In VVC Draft 8, when slice_ts_residual_coding_disabled_flag=1 both dependent quantization (DQ) and sign data hiding (SDH) can be used for TS blocks. Additionally, as reported in [Spec Ticket #859], though VTM-8.0 software disables the dequantization part of DQ for TS blocks it keeps the DQ related state-based context derivation.
In variant (#1a), it is proposed to disable both DQ and SDH for TS blocks. In addition to this variant, (variant #1b) further encoder fixes for RDOQ are provided for both BDPCM and non-BDPCM TS blocks when slice_ts_residual_coding_disabled_flag=1 as discussed in [VVC Ticket #981].
- For lossless coding on YUV420 sequences, the simulation results show overall bit-rate savings of -0.16% AI, -0.17% RA, and -0.28% LDB with Class F: -0.47% AI, -0.56% RA, and -0.65% LDB and Class TGM: -0.47% AI, -0.57% RA, and -0.60% LDB.
- For lossless coding on YUV444 and RGB sequences, the simulation results show overall bit-rate savings about -0.32% AI, -0.43% RA, and -0.46% LDB.
- For lossy coding without encoder fixes, the simulation results show overall BD-rate savings (luma) of -0.03% AI, -0.03% RA, and -0.03% LDB with Class F: -0.80% AI, -0.70% RA, and -1.02% LDB and Class TGM: -0.86% AI, -0.86% RA, and -1.32% LDB.
- For lossy coding after encoder fixes, the simulation results show overall BD-rate savings (luma) of -0.05% AI, -0.05% RA, and -0.10% LDB with Class F: -0.99% AI, -0.83% RA, and -1.21% LDB and Class TGM: -1.23% AI, -1.08% RA, and -1.80% LDB.
In variant #2, only DQ is disabled for TS blocks and in variant #3 only SDH is disabled for TS blocks. The results for variant #2 is the same as variant #1, no results were provided for variant #3 since SDH is disabled under CTC and encoder crash occurs when SDH is enabled in current VVC software as reported in Ticket #981.
In variant #4, it is proposed to align the current spec text to the VVC software, in which dequantization part of DQ is disabled for TS blocks as in current VVC software however DQ related state transitions and contexts are kept.
Discussed in session 2.6 Tuesday April 14 0850-0920 (chaired by JRO and BB)
Question: Is anything wrong with the current spec? The ticket #981 refers to software. There is a spec related ticket #859.
To achieve lossless coding, an encoder has to disable both DQ and SDH (and other things) at high level. This also applies for mixed lossy/lossless coding, which might then be performing worse than in case of local disabling these tools. Currently, disabling is possible at picture level.
Currently, the SW modifies DQ reconstruction for TS with RRC blocks locally, whereas the spec does not have such an element. Otherwise, it would not be possible to get lossless reconstruction. It is noted that when introducing the switch between RRC and TSRC, the modification of DQ reconstruction was never mentioned. The context derivation of DQ is retained.
Decision (cleanup/SW): It is pointed out that it might be better to clean up the software in this regard, even though due to disabling the combination of RRC with DQ at higher level this would never be used, it might be confusing for implementers. Experts from Qualcomm (original proponents from JVET-Q0089) volunteered doing the cleanup.
In the results for test 1 above, the anchor is not the CTC config for lossless, but a version that disables the context derivation of DQ as well as the reconstruction part. Results indicate that the SW mismatch (using context derivation of DQ) is not providing benefit.
There is no problem with TSRC, it is able to achieve lossless coding and there is no mismatch between spec and software.
An encoder could, with current spec, take the following options for mixed lossy/lossless
- Disable DQ, and enable RRC with TS, which would penalize the lossy coded parts
- Enable DQ, and disable RRC with TS, which would penalize the lossless coded parts (mainly for natural content, as per previous findings)
For lossless-only coding, there is no problem at all. For natural content, an encoder would just take the first choice if it is natural content.
No results are available for mixed lossy/lossless (as we don’t have CTC for this). Also, the current VTM encoder would need to modified, and realistic conditions (in terms of applications) are missing.
Aligning the text with software would introduce a block-level change that is not in the spec currently.
Another option would be a high-level restriction disallowing usage of DQ if TS/RRC is enabled (or the other way round). Also BDPCM should be considered in this context, as it can be enabled when TS is enabled.
It is mentioned that such a high-level restriction would be the cleanest approach with least danger of introducing even additional problems. See also further notes under JVET-R0119.
Issue was resolved by avoiding low-level changes - see further notes under JVET-R0271