JVET-M0109 CE12-related: block-based in-loop reshaping [E. François, C. Chevance, F. Hiron (Technicolor)]
This document describes a block-based implementation of the in-loop luma reshaping approach proposed in contributions JVET-L0246 and JVET-L0247. In-loop luma reshaping requires at the decoder two basic sample mapping operations: forward reshaping of the prediction signal, inverse reshaping of the reconstructed signal. Both operations are by default operated per sample, requiring a per-sample access to a look-up-table. In the proposed approach, the process is performed per block. For each 4x4 block, linear model parameters are computed from three luma values and their mapped version. The linear model is then applied to the block samples. The proposal results in PSNR-Y, U, V BD-rate variations compared to the VTM3.0 of -0.74%, 1.49%, 1.22% in AI configuration, -1.17%, 1.50%, 1.50% in RA configuration, -0.54%, 2.62%, 3.21% in LB configuration.
The described method does not perform quite as well as the per-sample look-up.
Some banding was reported for content with flat areas (esp. screen content).
Further work was encouraged to try to find some complexity reduction of the remapping process, although this scheme does not seem adequate as-is.
It was commented that for hardware it might actually be easier to do the per-sample processing, as this avoids the extra step of computing an average.