JVET-W0064 AHG8: Constraints on transforms and high precision operation [T. Ikai, T. Zhou, T. Hashimoto (Sharp)]
This contribution proposes to reduce complexity by utilizing exclusive usage on 64 point transform and LFNST for high bit depth coding. Specifically, 64 point transform and LFNST are normatively disabled when extended precision flag is equal to 1. In low QP HBD CTC experiment, the proposal shows up to 0.01% loss as follows.
- 0.04%, 0.06%, 0.05% in PQ
- 0.00%, 0.00% and -0.00% in HLG
- 0.00%, 0.00% and 0.00% in SVT
The more QPs condition from QP = -13 to 27 was also conducted. It shows proposed constraint (disabling 64 point transform and LFNST) show 0.37%, 2.12% and 2.16% loss in Y, U, and V at QP 12, 17, 22, 27 range in AI PQ 4:2:2. At the same time, this loss reduced to 0.11%, 0.42%, 0.27% at QP 7, 12, 17, 22 by enabling 64 point transform and disabling extended precision (this combination remains valid in this proposal).
Considering performance and complexity balance, this contribution recommends exclusive usage.
v2 fixed PQ simulation results to include FireEater for average bd-rate value in accordance to JVET-U2018.
v3 include alternative proposal where only LFNST is disabled when extended precision flag is equal to 1 as option1b to 3b. Also find the attached slide.
Summary results:
- Proposal
- Restrict 64 point transform and LFNST when extended precision is on
- 0.05% to 0.15% loss in QP= -13 to 27 in PQ AI (0.09% loss in HBT CTC)
- Alternative proposal
- Restrict LFNST when extended precision is on
- 0.06% possible loss in QP=12 to 27 in PQ AI (No loss in HBT CTC)
The main motivation is reduction of design changes relative to v1. However, design changes are anyway needed (e.g., disabling 64 point transform does not help much, as other transform sizes still need to be modified).
Disabling LFNST might help, but it also has small loss, and not obvious that the benefit is substantial – many other building blocks have to be redesigned, anyway.
No action was taken on this.