JVET-AL0212 AHG9/AHG15: Depth-adaptive picture scaling information SEI [V. Zakharchenko, T. Biatek, J. Boyce (Nokia)]

Discussed on 3 April 2025 at 17:20 chaired by S. Deshpande.

An SEI message is proposed to enable adaptive resampling of decoded picture based on corresponding samples of depth auxiliary pictures.

The Depth-adaptive picture scaling SEI message is asserted to be useful for use cases where the decoded picture must be resampled to a resolution of the display device to reduce a number of bitstreams stored for a variety of platforms, such use cases are broadcasting, teleconferencing, image/video analysis, reprojection, and creation of derivative content.

It was asked why the client cannot do the rescaling based on the original picture and depth map. The answer was that the SEI provides hint for rescaling with a prescribed algorithm.

It was asked- for the example rescaled images shown what was the scale factor and how the downscaled images look instead of the re-upscaled images. The answer was the images were down sampled by a factor of 8 in both dimension. The downscaled images were not easily available at this time.

It was asked if there are any results compared to naïve interpolation and how much is the benefit of the proposed approach. Not at this time, but they can be provided in future. It was asked what would be a good approach to compare these – PSNR or subjective metric. It was suggested to think about an appropriate metric to use for evaluation of results using the proposed approach.

It was asked if same algorithm is applied to chroma. It was answered that in the simulation the algorithm was applied separately to R, G, B. It was suggested to see how the results are for luma vs chroma and how the complexity differs in that case.

It was suggested to use depth map to have a weighted PSNR and calculate distortion. It was commented that possible problem of this approach is that bit allocation is done by importance and the importance assignment is standardized by SEI. The proponents rather think this as conveying artistic intent.

It was asked if block level scaling as proposed results in discontinuities. The answer was yes and smoothing filter on edges can be done. It was suggested to use overlap to handle this.

Further study recommended.