JVET-G0089 EE3 Related: Adaptive quantization for JEM-based 360-degree video coding [X. Xiu, Y. He, Y. Ye (InterDigital)]
Discussed Friday 17:45 (GJS & JRO).
This contribution presents an adaptive quantization method to enhance the performance of JEM-based 360-degree video coding. Compared to the HM-based adaptive quantization parameter (QP) methods in the current EE2, the main differences of the proposed method include: 1) it is proposed to independently apply different QP offsets for the luma and the chroma components for each coding tree unit (CTU); 2) it is proposed to use the input slice-level QP for the CTUs with the highest spherical sampling density and gradually decrease the QP value for the CTUs with lower spherical sampling density; 3) it is proposed to skip delta QP signaling, and let encoder/decoder derive and use the same QP adjustment. The simulation results are provided for the equirectangular projection (ERP) and cubemap (CMP) projection formats. It is reported that compared to the anchors based on JEM-6.0-360Lib-3.0, the proposed method provides on average {Y, Cb, Cr} gains of {5.0%, 6.8%, 6.8%} for the ERP and {2.6%, 11.2%, 12.4%} for the CMP in term of end-to-end WS-PSNR.
See also section 7.4.1 for a related discussion.
In this proposal, no signalling overhead is used. The decoder adjusts the QP automatically based on awareness of the mapping type.
It was commented that the signalling overhead for explicit signalling should be very small (in HM, it is 0.1% bit rate overhead for the signalling), so these results should similarly apply in that case (if the encoder is working properly). It is also mentioned by experts that more flexibility of QP control would be desirable e.g. for rate control.
Action item: Software coordinators of JEM and 360lib are mandated to resolve the issues with explicit signalling of QP adaptation. If it is demonstrated that adaptive QP does not cause visual artefacts, this should be implemented in JEM anchors.
BD-rate performance of the proposed adaptive quantization method for the ERP, compared to the JEM-6.0-360Lib-3.0 anchor.
SPSNR-NN (End to End) | SPSNR-I (End to End) | CPP-PSNR (End to End) | WS-PSNR (End to End) | |||||||||
Y | U | V | Y | U | V | Y | U | V | Y | U | V | |
Trolley | −2.0% | −2.4% | −5.0% | −2.0% | −2.4% | −5.0% | −2.0% | −2.4% | −5.0% | −2.0% | −2.4% | −4.9% |
GasLamp | −2.0% | −1.3% | 3.6% | −2.1% | −1.5% | 3.5% | −2.1% | −1.4% | 3.6% | −2.1% | −1.3% | 3.7% |
Skateboarding_in_lot | −9.6% | −19.4% | −17.6% | −9.6% | −19.3% | −17.6% | −9.6% | −19.3% | −17.6% | −9.5% | −19.3% | −17.5% |
Chairlift | −8.0% | −7.6% | −7.7% | −8.0% | −7.6% | −7.8% | −7.9% | −7.6% | −7.8% | −7.9% | −7.5% | −7.8% |
KiteFlite | −3.5% | −3.7% | −6.3% | −3.5% | −3.8% | −6.3% | −3.5% | −4.0% | −6.4% | −3.5% | −3.9% | −6.4% |
Harbor | −1.6% | −4.6% | −4.5% | −1.6% | −4.6% | −4.5% | −1.6% | −4.7% | −4.6% | −1.6% | −4.6% | −4.5% |
PoleVault | −7.9% | −10.5% | −11.4% | −7.8% | −10.2% | −11.2% | −7.8% | −10.2% | −11.2% | −7.8% | −10.6% | −11.4% |
AerialCity | −5.4% | −5.3% | −5.8% | −5.4% | −5.5% | −5.9% | −5.3% | −5.4% | −5.8% | −5.4% | −5.3% | −5.8% |
DrivingInCity | −0.8% | 1.4% | 2.5% | −0.8% | 1.3% | 2.4% | −0.7% | 1.3% | 2.3% | −0.7% | 1.5% | 2.5% |
DrivingInCountry | −9.3% | −14.2% | −15.8% | −9.2% | −14.3% | −15.8% | −9.2% | −14.4% | −15.9% | −9.3% | −14.3% | −15.8% |
Overall | −5.0% | −6.8% | −6.8% | −5.0% | −6.8% | −6.8% | −5.0% | −6.8% | −6.8% | −5.0% | −6.8% | −6.8% |
BD-rate performance of the proposed adaptive quantization method for the CMP, compared to the JEM-6.0-360Lib-3.0 anchor
AQP_CMP vs. CMP | SPSNR-NN (End to End) | SPSNR-I (End to End) | CPP-PSNR (End to End) | WS-PSNR (End to End) | ||||||||
Y | U | V | Y | U | V | Y | U | V | Y | U | V | |
Trolley | −2.7% | −8.2% | −7.5% | −2.6% | −8.3% | −7.6% | −2.6% | −8.2% | −7.4% | −2.6% | −8.1% | −7.4% |
GasLamp | −2.2% | −10.1% | −10.8% | −2.2% | −10.1% | −10.9% | −2.3% | −10.0% | −10.8% | −2.3% | −9.9% | −10.8% |
Skateboarding_in_lot | −2.8% | −12.2% | −13.9% | −2.8% | −12.3% | −13.9% | −3.0% | −12.4% | −14.1% | −3.0% | −12.4% | −14.1% |
Chairlift | −3.8% | −13.4% | −12.2% | −3.8% | −13.4% | −12.2% | −3.7% | −13.2% | −12.1% | −3.7% | −13.1% | −12.1% |
KiteFlite | −1.9% | −10.3% | −8.4% | −1.9% | −10.3% | −8.5% | −1.9% | −10.4% | −8.5% | −1.9% | −10.3% | −8.4% |
Harbor | −2.1% | −7.8% | −8.3% | −2.0% | −7.8% | −8.3% | −2.2% | −8.0% | −8.4% | −2.1% | −7.9% | −8.4% |
PoleVault | −1.4% | −10.7% | −14.0% | −1.3% | −10.5% | −13.8% | −1.4% | −10.6% | −13.8% | −1.4% | −10.8% | −14.1% |
AerialCity | −2.9% | −13.9% | −12.4% | −2.9% | −14.0% | −12.5% | −3.0% | −14.1% | −12.3% | −3.0% | −13.9% | −12.3% |
DrivingInCity | −2.3% | −9.7% | −9.3% | −2.3% | −9.8% | −9.3% | −2.6% | −9.9% | −9.6% | −2.5% | −9.8% | −9.5% |
DrivingInCountry | −3.2% | −15.8% | −17.7% | −3.4% | −15.9% | −17.7% | −3.2% | −15.6% | −17.5% | −3.0% | −15.5% | −17.5% |
Overall | −2.5% | −11.2% | −11.5% | −2.5% | −11.2% | −11.5% | −2.6% | −11.2% | −11.5% | −2.6% | −11.2% | −11.4% |
Additionally, similar to the tests in EE3, supplemental simulations were conducted by applying the proposed adaptive quantization method on the rotated ERP pictures where the content that contains detailed texture/edges are rotated from the equator to the pole. To rotate the input ERP pictures, the “SVideoRotation” parameter in the encoder configure file is set equal [0 90 0]. The table below presents the BD-rate savings in terms of the end-to-end distortion measurements of the proposed method for the rotated ERP.
BD-rate performance of the proposed adaptive quantization method for the rotated ERP, compared to the rotated ERP coded by JEM-6.0-360Lib-3.0
AQP_ERP_ROT vs. ERP_ROT | SPSNR-NN (End to End) | SPSNR-I (End to End) | CPP-PSNR (End to End) | WS-PSNR (End to End) | ||||||||
Y | U | V | Y | U | V | Y | U | V | Y | U | V | |
Trolley | −6.8% | −7.7% | −3.8% | −6.8% | −7.8% | −3.9% | −6.7% | −7.7% | −3.7% | −6.7% | −7.6% | −3.7% |
GasLamp | −6.6% | −6.4% | −8.2% | −6.6% | −6.6% | −8.4% | −6.4% | −6.5% | −8.3% | −6.4% | −6.4% | −8.2% |
Skateboarding_in_lot | −9.4% | −8.0% | −7.1% | −9.4% | −8.1% | −7.1% | −8.9% | −7.4% | −6.4% | −9.0% | −7.4% | −6.4% |
Chairlift | −10.4% | −14.4% | −14.5% | −10.4% | −14.4% | −14.6% | −10.3% | −14.5% | −14.7% | −10.4% | −14.5% | −14.7% |
KiteFlite | −8.3% | −8.3% | −6.8% | −8.3% | −8.4% | −6.9% | −8.0% | −7.9% | −6.4% | −8.0% | −7.9% | −6.3% |
Harbor | −11.5% | −12.8% | −13.8% | −11.6% | −12.8% | −13.8% | −11.3% | −12.7% | −13.5% | −11.4% | −12.6% | −13.5% |
PoleVault | −11.0% | −15.2% | −17.0% | −11.3% | −15.3% | −17.0% | −11.1% | −15.0% | −16.7% | −10.9% | −15.1% | −16.7% |
AerialCity | −8.8% | −11.5% | −12.5% | −8.9% | −11.8% | −12.7% | −8.5% | −11.4% | −12.6% | −8.5% | −11.3% | −12.3% |
DrivingInCity | −8.9% | −10.5% | −9.9% | −9.0% | −10.7% | −10.2% | −8.3% | −9.9% | −9.4% | −8.4% | −9.9% | −9.5% |
DrivingInCountry | −8.3% | −13.6% | −6.2% | −8.7% | −13.7% | −6.5% | −8.5% | −13.6% | −6.3% | −8.2% | −13.5% | −6.1% |
Overall | −9.0% | −10.8% | −10.0% | −9.1% | −11.0% | −10.1% | −8.8% | −10.7% | −9.8% | −8.8% | −10.6% | −9.7% |