JVET-D0074 AHG8: Suggested test methodology for evaluating projection formats for 360 video coding [B. Vishwanath, Y. He, X. Xiu, Y. Ye (InterDigital)]
At the 3rd JVET meeting, AHG8 on 360 video test conditions was established. One of AHG8’s mandates is to “study the effect on compression when different warping methods are applied to the input 360 video before compression.” A main challenge for the test is that the input video is always in Equirectangular Projection (ERP). This gives ERP an obvious edge over the other projection formats. In this document, a test methodology was proposed to compare different projection formats and their impact on 360 video coding, aimed at reducing/removing the bias towards ERP, the geometry in which all currently available original spherical content is represented. It wsa further proposed to also study the impact on rendered video quality within a given viewport using different projection formats in future JVET tests, as it was suggested that consuming 360 video content on conventional 2D displays will continue.
In this second revision, simulation results for compact octahedron projection (COHP) and compact icosahedron project (CISP) were additionally provided.
The contribution also proposes viewport based testing.
Tests of the impact on PSNR were provided of converting 8K ERP to various 4K representations and converting back to 8K ERP, and measuring with AW-sPSNR. This reportedly showed a content dependent impact on the conversion impact. Generally, ERP had the most reported PSNR loss, and ISP had the least PSNR loss. All 4K projection formats had the same number of active samples.
Then HM encoding/decoding was conducted in the projection format. The best format reportedly differed for different sequences.
Sequence | ERP4K (4096x2048) | CMP4K eq. (3552x2368) | EAP4K (4096x2048) | OHP4K eq. (6208x2688) | ISP4K (2470x3424) |
Basketball | 45.59 | 46.22 | 41.23 | 47.26 | 48.61 |
Chairlift | 47.40 | 48.05 | 44.42 | 48.71 | 50.21 |
Jam Session | 51.45 | 51.76 | 52.64 | 52.15 | 53.44 |
Skate in lot | 46.50 | 47.05 | 47.39 | 50.07 | 50.19 |
Skate Trick | 44.80 | 45.11 | 46.05 | 45.53 | 46.21 |
Train | 45.65 | 46.06 | 46.66 | 46.12 | 46.68 |
Dancing | 47.63 | 48.10 | 47.81 | 48.38 | 49.21 |
Driving | 48.35 | 48.98 | 49.89 | 51.12 | 51.52 |
Average | 47.17 | 47.67 | 47.01 | 48.67 | 49.51 |
It was suggested to avoid converting back to 8K for the PSNR calculation, and instead do it at 4K.
It was suggested to separate comparison of non-ERP projection formats with each other, and separately compare ERP with a non-ERP comparisons.
Viewport evaluation was described, with simulations of a single viewport with 60 x 60 degree front viewport. That might be a small degree range. It would be useful to gather information about what field of view ranges are supported in products that would be used for consuming the content.
Viewport generation in rectilinear format was considered. For a larger field of view, the rectilinear format has distortion at the edges.
There was a suggestion for testing to use viewport location that includes projection face edge.
Questions were raised in contribution: How many viewports should be evaluated? Where should we set them? Should we use dynamic viewports?
Additional questions included: For subjective testing, how will the viewport be viewed, e.g. on what device? Generation of the viewport includes interpolation, and quality will be impacted by the interpolation filters that are used.