JVET-AF0063 On MV-HEVC profiles [Y.-K. Wang, H. Liu, L. Zhang, S. Jiao, C. Hu, J. Cui, G. Xu (Bytedance)]

It was asserted that, although the syntax of MV-HEVC allows for an output layer set (OLS) in an MV-HEVC bitstream to have both texture layers and depth layers, the profile definitions do not allow this. Recently, applications have come up, e.g., virtual reality, that benefit from all the following three aspects simultaneously: 1) using inter-view prediction between different views, 2) having depths associated with the coded views to be available to the decoder and the renderer, and 3) using existing HEVC hardware encoder/decoder designs. It is also asserted that in such applications it is desirable that the bit depth of the depth component can be of up to 16 bits.

This contribution proposes the following:

1. Update the definitions of the four new profiles, namely the Multiview Main 10, Multiview Monochrome, Multiview Monochrome 10, and Multiview Monochrome 12 profiles, to allow for an OLS containing a layer conforming to such a profile to have both texture layers and depth layers.
2. Add a new profile, named the Multiview Monochrome 16 profile, where it is also allowed for an OLS containing a layer conforming to this profile to have both texture layers and depth layers.

The proposed changes, including some minor editorial changes not summarized above, are included in an attachment, with all the proposed changes marked relative to JVET-AE1006-v1.

It was reported that the existing definition of the new Multiview profiles does not support carriage of depth maps. A text is provided to release the constraint in the profile definition. In that context, the bit depth of the depth map can be larger than the bit depth of the texture. There are also a number of additional editorial improvements.

The question was raised whether also a new 8-bit Multiview profile should be defined that supports carriage of depth. It was later found that this is not necessary, as the wexisting profile already allows depth (see below).

After offline discussion between A. Tourapis, I. Moccagatta, and Y.-K. Wang, a new version of the contribution was produced which was presented on Thu. 19 Oct. at 1230.

The following aspects were newly observed:

1. The multilayer extensions of HEVC allow two ways of representing depth:
   1. Via DepthLayerFlag  = =  1 (depth), and/or
   2. Via AuxId  = =  AUX_DEPTH (auxiliary depth)

And both of the following combinations are allowed:

1. DepthLayerFlag  = =  1  &&  AuxId  = =  AUX_DEPTH
2. DepthLayerFlag  = =  0  &&  AuxId  = =  AUX_DEPTH
3. Although the current definitions of the existing Multiview Main profile and the being-developed Multiview Main 10 profile do not allow having both texture layers and depth layers in an OLS, they do allow having both auxiliary depth layers with chroma_format_idc equal to 1 and texture layers in an OLS but do not allow having both auxiliary depth layers with chroma_format_idc equal to 0 and texture layers in an OLS (referred to as “the wanted combination” below for simplicity).
4. Identified another asserted bug for the profile definitions in the CDAM text, on the Multiview Main 10 profile: missing the constraint of colour_mapping_enabled_flag equal to 0 only.

In JVET-AF0063-v2, instead of proposing to allow having both texture layers and depth layers in an OLS, it is proposed to allow the wanted combination. Two options summarized as follows are proposed:

1. Update the being-developed Multiview Main 10 profile to allow the wanted combination.
2. Keep the being-developed Multiview Main 10 profile disallowing the wanted combination, but add a second Multiview Main 10 profile and also a second Multiview Main profile that both allow the wanted combination.

Besides, both options also include the addition of the Multiview Monochrome 16 profile and updates to the being-developed Multiview Monochrome, Multiview Monochrome 10, and Multiview Monochrome 12 profiles, and all these monochrome profiles allow the wanted combination.

It was commented that the design of the existing 8-bit Multiview profile was done as it is (coding depth in a 4:2:0 fashion without filling the chroma channels with any information), because existing hardware could right away be used for decoding texture and depth.

It was argued by the proponent that in the 10 bit domain, most decoding hardware (consumer sector) would support monochrome. This however does not seem to be the case, according to other experts’ opinion. There are so many HEVC Main 10 decoder chips in the market, that it is difficult to assess, anyway.

For 12 bit, this situation would be different, as in range extensions monochrome is implicit.

Defining two different 10 bit Multiview profiles is inappropriate. Furthermore, the unnecessary processing of not-used chroma channels in 4:2:0 is not a heavy burden, even less if depth maps are downsampled in practical texture+depth applications

It was concluded to keep the definition for the Multiview Main 10 profile as it is in terms of encoding depth maps. This also retains the same concept as in the 8 bit profile.

JVET-AF0063 On MV-HEVC profiles [Y.-K. Wang, H. Liu, L. Zhang, S. Jiao, C. Hu, J. Cui, G. Xu (Bytedance)]

See notes in section 4.2.

General aspects of standards development and applications of standards (1)

Contributions in this area were discussed at 1735–1745 on Thursday 19 Oct. 2023 (chaired by JRO).