JVET-P0687 AHG17: A summary of HLS proposals on access unit delimiter, picture header, and slice header parameters signalling [Hendry (Futurewei)] [late]

Discussed Wednesday 2030 (GJS & JRO)

This contribution provides a summary of high-level syntax proposals submitted to this JVET meeting related to access unit delimiter, picture header, and slice header parameters signalling. A list of design questions is provided to help structure the review of the proposals.

The contributions in this category can be classified into two sub-categories:

• AUD and Picture Header
• Modifications to constant slice header paramater signalling (related to constant_slice_header_params_enabled_flag)

General questions:

1. Do we want to avoid signalling syntax elements in slice header that are constrained to have same values for all slices of a picture?

If yes. Where to signal them? AUD? New NAL Picture Header? Use the constant slice header params signalling in PPS?

1. For constant slice header parameters, should some of the syntax elements be in PPS? or in AUD? or in Picture Header, if exists?

On AUD and Picture Header.

1. Do we want to have Picture Header?

Proposing yes. JVET-P0095, JVET-P0120, JVET-P0239

Why? JVET-P0092 & JVET-P0120: AUD is AU specific. An AU may contain multiple pictures.

JVET-P0120: Order of AUD is supposed to the first NAL unit in an AU. If AUD preceeds SPS & PPS, some picture-level syntax elements in SH cannot be moved to AUD. Having PH is asserted would solve this problem.

Decision: Yes, after considering the quantity of syntax elements that would go there, we should have a picture header.

1. If yes, what to signal in PH?
   1. Move some signalling from PPS to PH

JVET-P0095: Subpicture Id mapping, if it is not signalled in the SPS and not signaled in the PPS, to avoid needing to send a PPS for this

Decision: Allow the mapping in the PH when it is not in the SPS. If present, it overrides what is in the PPS. If not in the PPS and not in the SPS, then is shall be in the PH. As detailed below:

SPS syntax:

fancy_weird_stuff_allowed_flag

if( fancy_weird_stuff_allowed_flag ) {

sending_details_in_sps_flag /* infer 0 if not present */

if ( sending_details_in_sps_flag )

Details

}

PPS syntax:

sending_details_in_pps_flag

if (sending_details_in_pps_flag)

Details

constraint: If fancy_weird_stuff_allowed_flag is 0 or sending_details_in_sps_flag is 1, sending_details_in_pps_flag shall be equal to 0.

PH syntax:

if ( fancy_weird_stuff_allowed_flag && !sending_details_in_sps_flag ) {

sending_details_in_ph_flag

if(sending_details_in_ph_flag)

Details

}

• 1. Signal syntax element both in PPS and PH. Present in PH if not present in PPS

JVET-P0095: Tile/Brick/Rect Slice signalling

Deferred to after considering GDR needs.

• 1. Move some signalling from SH to PH

Preliminary decision:

• • 1. JVET-P0095, JVET-P0120: no_output_of_prior_pics_flag
    2. JVET-P0120, JVET-P0239: slice_pic_parameter_set_id, non_reference_picture_flag, colour_plane_id, slice_pic_order_cnt_lsb, recovery_poc_cnt, pic_output_flag, slice_temporal_mvp_enabled_flag.

Comment: Think more about slice_pic_order_cnt_lsb as a loss detection mechanism

• • 1. JVET-P0239: slice_lmcs_aps_id and slice_scaling_list_aps_id
    2. JVET-P0239 move some syntax elements that are currently not constrained to be the same for all slices of a picture, but asserted having no / minimal benefit and coding loss to transmitting them in every slice header as their anticipated usage would change at the picture level. 18 such syntax elements are identified in the contribution. (In the CTC, about half of these don't even change across different pictures.)

It was commented that this could affect the ability to merge slices from different encoders. However, others commented that encoding for such applications would already be customized to this use case (and possibly only coming from one common encoder or encoders from only one company).

• • • 1. six_minus_max_num_merge_cand
      2. five_minus_max_num_subblock_merge_cand
      3. slice_fpel_mmvd_enabled_flag
      4. slice_disable_bdof_dmvr_flag
      5. max_num_merge_cand_minus_max_num_triangle_cand
      6. slice_six_minus_max_num_ibc_merge_cand
      7. partition_constraints_override_flag
      8. slice_log2_diff_min_qt_min_cb_luma
      9. slice_max_mtt_hierarchy_depth_luma
      10. slice_log2_diff_max_bt_min_qt_luma
      11. slice_log2_diff_max_tt_min_qt_luma
      12. slice_log2_diff_min_qt_min_cb_chroma (only needed for I slices)
      13. slice_max_mtt_hierarchy_depth_chroma (only needed for I slices)
      14. slice_log2_diff_max_bt_min_qt_chroma (only needed for I slices)
      15. slice_log2_diff_max_tt_min_qt_chroma (only needed for I slices)
      16. mvd_l1_zero_flag (only needed for B slices)
      17. dep_quant_enabled_flag
      18. sign_data_hiding_enabled_flag
  1. Signal some syntax element that are not constrained to be the same for all slices of a picture both in PH and SH. Present in SH if not present in PH
     1. JVET-P0120: RPL, joint cb/cr sign flag, SAO, ALF, LMCS, Scaling list

Preliminary decision:

joint cb/cr sign flag should be in the PH and never be in the SH

RPL, SAO, ALF, LMCS, deblocking (4 syntax elements in July output doc) should possible to put in SH (as override if also in higher layer), also possible in PH and in SPS (for each of these five categories, if it's in the PH, then it's not in the SH)

Scaling list control should never be in the SH

• 1. New syntax elements:
     1. JVET-P0095: A flag to specify whether associated picture contains mixed NAL unit types (i.e, mixed_nalu_types_in_pic_flag). See JVET-P0124, JVET-P0146, and JVET-P0222.

See the notes for the HLS BoG.

• • 1. JVET-P0120: Picture type (whether picture is IDR, CRA, GDR, contains only I-slices, only P- and I- slices, any type of slices)

See the notes for the HLS BoG.

Discussion continued Track B Thursday 3 October 1115 (GJS).

1. If yes, what is property of PH?
   1. JVET-P0095, JVET-P0120, JVET-P0239: Mandating one PH one picture

Decision:

PH comes before the VCL NAL units for the picture; the SPS and PPS must precede it. APSs do not need to precede it but may. SEI messages for the picture follow it.

Do not allow PH to be "out of band" (and it should be in Type I category of HRD).

Allow repetitions; not more PHs than slices; not after the last VCL NAL unit.

• 1. JVET-P0120: Specifies layer Id, temporal Id, order of PH in access unit, etc.

Decision: Layer Id and temporal Id are those of the picture.

• 1. Is it VCL or non-VCL?

Decision: Non-VCL

• • 1. JVET-P0095, JVET-P0120: PH is a non-VCL NAL unit.
    2. JVET-P0239: PH is a VCL NAL unit.

1. Do we want to keep AUD?

Proposing no: JVET-P0095, JVET-P0120

Decision: Optional AUD (restore language of boundary detection). See also next item.

1. Do we want to keep mandating the presence of AUD?

Proposing no:

JVET-P0218. Signal single_slice_in_pic_flag in SPS. When single_slice_in_pic_flag is equal to 1, AUD is not mandated to be present. single_slice_in_pic_flag can be used to condition the presence of other syntax elements such as num_bricks_in_slice_minus1.

JVET-P0367: Signal a flag aud_in_next_access_unit_present_flag in AUD to specify whether the AUD is required to be present for next AU.

JVET-P0480: AUD is mandated only when there are more than 1 subpicture. Signal first_slice_in_sub_pic_flag in SH. This flag can be used for detecting first slice of each picture when there is only one subpicture in a picture.

Conclusion: See above item 5 (AUD optional).

1. Do we want to signal more syntax elements in AUD?
   1. JVET-P0218: Optionally moves PPS Id, non_reference_picture_flag, colour_plane_id, slice_pic_order_cnt_lsb, and pic_output_flag to AUD, if present. Create a syntax structure pic_header(). NOTE: pic_header is not in its own NAL unit.
   2. JVET-P0146: Signal syntax elements which specifies types of subpictures of a picture (non-IRAP, IDR, CRA). This would allow a picture to contain mixed subpicture types.
   3. JVET-P0222: Signal rap_type and au_order_cnt. Syntax element rap_type specifies which specifies whether the coded pictures in the access unit contain non-IRAP NAL unit only, IRAP NAL unit only, or both IRAP and non-IRAP.

Comments: JVET-P0146 and first aspect of JVET-P0222 suggest to allow mixed of IRAP and non-IRAP in a picture. They should be discussed together with JVET-P0095 and JVET-P0124.

No action was taken on this.

1. AUD properties:

Decision: Adopt points a and c below.

• 1. JVET-P0218: The AUD shall have nuh_layer_id value equal to vps_layer_id[ 0 ].
  2. JVET-P0380: The AUD shall have nuh_layer_id value equal to 0.
  3. JVET-P0218: The AUD shall have TemporalId equal to the TemporalId of the access unit containing the NAL unit

On signalling constant slice header params.

1. Some bugs are asserted to be present in the signalling of syntax element when constant_slice_header_params_enabled_flag? See the notes for the HLS BoG report.
   1. JVET-P0152:
      1. Update semantics of pps_five-minus_max_mun_subblock_merge_cand_plus1 to incorporate the value of sps_affine_enabled_flag.

Comment: The proposed changes seem to be more appropriate to be expressed as constraints.

• • 1. Update the inference of the value five_minus_max_num_subblock_merge_cand in SH when it is not present taking into account the proposed change above.
  1. JVET-P0427 proposal 1: Condition the signalling of pps_max_num_merge_cand_minus_max_num_triangle_cand_plus1 on the value of pps_six_minus_max_num_merge_cand_plus1
  2. JVET-P0427 proposal 2: Update the range value of pps_max_num_merge_cand_minus_max_triangle_cand_plus1 to not depend on the value of MaxNumMergeCand
  3. JVET-P0427 proposal 3: MaxNumTriangleMergeCand is inferred to be equal to 2 in slice header if MaxNumMergeCand is equal to 2 without signalling in either PPS or slice headers
  4. JVET-P0605: proposed to modify pps_max_num_merge_cand_minus_max_num_triangle_cand_minus1 to pps_max_num_merge_cand_minus_max_num_triangle_cand_plus1

This has been addressed by editor between last meeting and this meeting. The update is reflected in JVET-P0113

1. Adding more syntax element to be signalled in PPS when constant_slice_header_params_enabled_flag is equal to 1:
   1. JVET-P0334 proposal / modification 1
   2. JVET-P0368
   3. JVET-P0428 proposal 1

Discussed Thursday Track B 1300 (GJS).

With us now having a PH, it was suggested for this to be a matter of indicating whether something is in the PPS or the PH. Whereas previously this mechanism would be saving one instance of each syntax element per slice, it would now be saving only one instance per picture header. This aspect was agreed in principle, with further discussion of the details.

See the notes for the HLS BoG report.

1. Improvement for signalling mechanism of constant slice header params:
   1. JVET-P0334 proposal 2 & 3:
      1. Instead of signalling 1 set of constant slice header params in PPS, allow signalling up to 7 sets of constant slice header params in PPS. In SH, signal index of the set to be used.
      2. When the number of constant slice header params set in PPS is equal to 0, signal the syntax elements in SH
   2. JVET-P0428 proposal 2: signal conformance_constant_params_enabled_flag in SH to specify if the slice uses the constant syntax elements signalled in PPS or not. This allow constant slice params to be present in the PPS but also present in SH (i.e., when the value of conformance_constant_params_enabled_flag is equal to 0)