JVET-AD0175 AHG8/AHG9: On object mask auxiliary picture and object mask information SEI message for VSEI and HEVC [J. Chen, S. Wang, Y. Ye (Alibaba)]

This was to be discussed Tuesday morning together with SEI experts.

JVET-AD0175 AHG8/AHG9: On object mask auxiliary picture and object mask information SEI message for VSEI and HEVC [J. Chen, S. Wang, Y. Ye (Alibaba)]

This contribution also fits under section 4.10.

In this contribution, it is proposed to add a new type of auxiliary picture to represent the object mask for the detected or tracked object within the compressed video bit-stream in VSEI and HEVC. To support the proposed object mask auxiliary picture, the associated SEI message called the object mask information (OMI) SEI message is also proposed to VSEI and HEVC.

The proposal is to encode the aux picture in 10 bits, and use the value as index for the corresponding object (max. 1024). There could be multiple auxiliary pictures.

Questions asked about this included:

• Is lossless coding necessary, and what could be data rates?
• What would be typical use cases of having 1024 different objects?
• Could the auxiliary pictures be downsampled?
• Could the auxiliary picture also apply to a cropped part? It was commented that the cropping window cannot be used for this purpose, as this is intended for display.

Further study was recommended along these questions.