JVET-AQ0204 How to calculate average PSNR for a video sequence [J. Samuelsson-Allendes, S. Deshpande (Sharp)] [late]
This contribution relates to the calculation of average PSNR values for a video sequence. Two methods are compared;
- Picture-based PSNR (P-PSNR for short) where a PSNR value is calculated for each picture and the average is calculated across those PSNR values
- Sequence-based PSNR (S-PSNR for short) where the MSE is calculated across all sample values in the sequence and the PSNR value for the sequence is calculated from the sequence level MSE
The two methods are compared with a literature analysis, an industry practice analysis, a logical analysis, an empirical analysis and an evaluation on CTC data (for both codec-level and tool-level).
The proposal is to retire the use of P-PSNR and instead start using S-PSNR whenever PSNR numbers are reported for video sequences and when calculating BD-Rate numbers. It is suggested to perform the shift to S-PSNR before the evaluation of CfP responses and then use it consistently in the standardization phase of the upcoming video coding standard.
An analysis shown during the presentation indicates that the BD rate result ECM vs. VTM is almost identical for both types of PSNR. On tool level, temporal filter and LMCS were the only ones showing major differences (LMCS mainly for one sequence, MarketPlace).
It was commented that an extreme case where P-PSNR is wrong would be a black frame with zero MSE. Also sequences with scene cuts and PSNR largely different in the two parts are problematic. In particular, might also be problematic for screen content.
Conversion from data of picture-wise PSNR with 2-digit precision to MSE and from that to S-PSNR is possible.
Various experts also expressed preference for S-PSNR. It was however argued that any PSNR is still far away from visual quality.
One argument pro P-PSNR is that it better reflects the advantage of QP modifications of B hierarchies in terms of visual quality. This should however not be over-stressed as it may improve pumping. In comparing tools when using same temporal structure it may not be relevant which version to use.
It was generally agreed that it is a good point in time to move from P-PSNR to S-PSNR when entering development phase of a new video compression standard. Further investigation should be conducted until the next meeting based on the analysis script that is planned to be used, to study whether the precision foreseen for the frame-wise PSNR with the .csv files is sufficient for conversion into S-PSNR.
It is noted that further study is necessary if the switch to S-PSNR is also beneficial for WPSNR used for PQ content.
AHG18 Ultra-low latency and packet loss resilience (2)
Contributions in this area were discussed during 1220–1300 on Friday 10 July 2026 (chaired by JRO).