JVET-N0246 Modified dequantization scaling [K. Sharman, S. Keating (Sony)]
In VVC WD4, when the size of a transform block is not a power of 4, the transform coefficients are modified by a √2 factor, to compensate for an implicit scaling by the transform process.
It is proposed that the scaling be applied by a modification to the QP or QP levelScale table rather than by multiplication by 181/256 (or 181/128).
The proposal reportedly produces negligible BD-rate changes. It is a continuation of JVET-M0119, and is independent, but related to JVET-L0095.
Initial presentation Thu 21 evening, discussion continued Fri 22 11:30
Intent is simplification relative to the current method
6 different methods are proposed.
In the discussion, a certain preference is expressed to use either one of the approaches with 2 tables (methods 3/4, or (which is almost equivalent) to implement those as /2 or x2 (methods 5/6).
All differences in terms of performance are irrelevant. It is agreed to adopt method 6.
Decision: Adopt JVET-N0246, method 6.
Entropy coding (8)
Contributions in this category were discussed Friday 22 March 1150-1350 (Track A chaired by JRO).