JVET-V0066 AHG8: Encoder improvements to palette coding for high bit depth [T. Tsukuba, M. Ikeda, T. Suzuki (Sony)]
This contribution proposes two encoder modifications to palette coding for high bit depth: Proposal#1 uses on-the-fly bit calculation functions instead of pre-defined lookup tables to reduce memory usage and Proposal#2 fixes QP offset derivation used for an error limit to improve coding performance of palette coding. It is reported that Proposal#1 has no impact to coding performance and can reduce memory size. Regarding Proposal#2, it is reported that average bd-rate changes for YUV at normal QP are (-8.70%, -10.86%, -11.66%) for AI and (-6.09%, -8.43%, -9.16%) for RA, and average bd-rate changes for RGB are (-8.51%, -8.65%, -8.32%) for AI and (-5.64%, -5.86%, -5.73%) for RA; average bd-rate changes for YUV at low QP are (-1.80%, -2.08%, -2.09%) for AI and (-1.34%, -1.27%, -1.31%) for RA, and average bd-rate changes for RGB are (-1.87%, -1.93%, -1.96%) for AI and (-1.19%, -1.33%, -1.27%) for RA, respectively.
Proposal2 is obvious bug. It is asked if Proposal1 would have impact on CTC conditions in terms of run time. It would be desirable to replace the lookup table by on the fly computation for all bit depth cases. Proponents were asked to investigate on this aspect. It was later confirmed that there is no impact when the on-the-fly computation is exercised in CTC 4:4:4 condition, neither on runtime nor on performance.
Decision (SW/BF): Adopt JVET-V0066 both aspects. The method should replace the lookup table approach (which also removes some macros) and be uniquely applied for both “normal” CTC and high bit depth.