JVET-J0044 AHG8: Geometry padding for PERP [P. Hanhart, Y. He, Y. Ye (InterDigital)] [late]
This contribution was discussed Thursday 19 April at about 1310 (chaired by GJS & JRO).
The padded ERP (PERP) format was shown to effectively reduce the seam artefacts in reconstructed viewports that encompass the left and right boundaries of the ERP picture (see JVET-G0098). Therefore, the PERP format was selected as the format used for the HM and JEM anchors for the 360° category in the joint CfP. However, padding and blending may not be sufficient to completely resolve the seam issue, as a seam artefact is still visible in the HM and JEM anchors for the Chairlift sequence. It was reported that the seam artefact in the Chairlift test sequence is particularly visible during video playback, as the video appears to flicker where face seams occur.
Geometry padding of the reference pictures was previously proposed for motion compensated prediction (JVET-D0075). Compared to the repetitive padding method used in HEVC, geometry padding can provide meaningful samples and improve the continuity of neighbouring samples for areas outside of the reference picture boundaries. For HM-16.12 and 8K ERP sequences, average BD-rate reductions of 0.3% and 0.8% were previously reported for the ERP and cubemap formats, respectively.
This proposal investigates the performance of geometry padding for motion compensated prediction when encoding in the PERP format. The simulation results using the CfP settings reportedly show that geometry padding achieves average BD-rate reduction of 0.23%, with up to 0.98% savings for Chairlift, at a similar complexity as the JEM. More significantly, it is reported that geometry padding combined with the PERP format completely solves the seam artefact problem. It was reported that no seam artefact was visible in the contributor's expert viewing for the reconstructed viewports located near the PERP boundaries when the geometry padding was used.
It was reported that for 360° sequences with fast-moving camera, the boundary is still visible. Geometry padding (used in addition to padding of 8 samples at each side) can reportedly resolve this. It was reported that this combination (plus blending) helped to remove this effect.
It had been suggested by other experts prior to the presentation that extending the padding area in conventional padding could help, but it was reported that this did not fully remove the artefacts, and further caused bit rate increase.
Further, the same as with geometry padding approaches proposed in the context of other projection formats, the decoder would need to know where the boundary is, and it would need to know the manner of continuation at the other side.
The anchor used in the CfP used 8-sample padding with linear blending across the boundary. The contributor said that using a wider band, 32 samples, seems to largely eliminate the seam artefact, but produces some reduced or blurred appearance in the blended region and has a bit rate overhead of about 1%. Using a padding of 16 samples produces something in between, with perhaps a 0.3% bit rate penalty.
The proponent said they could show the effect on viewing equipment available at the meeting.
It was commented that the standard could specify modulo wrapping by the picture size for picture location referencing for the ERP projection mapping, which could be supported in a decoder by padding with the maximum block width and modulo wrapping of the block vectors.
For other projection formats, there would be more complicated implications of this sort of wrapping.
It was recommended for this to be further studied in an AHG on 360° video coding.