JVET-AL0286 AHG9: On dual-lens optical correction SEI message [A. Karabutov, E. Alshina (Huawei)] [late]

The contribution proposes add support of the case of projection of images from dual-lens fish-eye systems on the same frame. Specifically, the following changes relating to the optical correction SEI message are proposed:

1. Signalling the number of projection of images on the same frame by the syntax element “loc_number_minus1”.

2. Signalling the optical parameters for each lens separately.

3. Signalling whether the second lens has the same direction as the first lens by the flag “loc_inverse_direction”.

The contribution notes that there are various camera configurations available on the market that employ dual-lens systems or feature two sensors, where images captured by both lenses or sensors are combined and stored within the same frame.

The proposed loc_number_minus1 syntax element is u(1), so only allows 1 or 2 lenses. It should maybe be renamed as a flag. It was suggested to consider more than 2 lenses.

As compared to the Multiview acquisition info SEI message, this contribution is aimed at content where a single picture contains the results of capture by two fisheye lenses. No information is provided about where in the coded picture each lens capture is represented.

Semantics of the loc_inverse_direction[ i ] need some improvement to clarify what opposite direction means.

It was noted that there is a fisheye video info SEI message in the omnidirectional SEI message category in HEVC that describes content captured by a device with two fisheye lenses aimed in opposite directions. The fisheye video info SEI message is not in VSEI.

Some support expressed for the proposed functionality.

It was suggested that it would be useful to see a showcase and software for the proposal.

Further study encouraged to improve semantics and distinguish capability vs. existing the fisheye video info SEI message in HEVC, and consider if more than 2 lenses should be supported.