JVET-R0406 AHG9: A summary of proposals on syntax for one slice per picture [Y.-K. Wang (Bytedance)]

Discussion began here for AHG Session 1.14 on Monday 13 April at 0500 UTC (chaired by GJS & YKW).

This contribution intends to provide a summary of the 13 proposals on syntax for one slice per picture submitted to this JVET meeting.

It is suggested that this summary, in terms of a list of design questions, is used for the reviewing of these proposals, such that the discussions can be in a more structured and efficient manner.

1. Add an SPS flag sps_one_slice_per_picture_flag (or a different name with the same semantics: sps_picture_header_in_slice_header_flag). (JVET-R0060, JVET-R0118)
   1. When sps_one_slice_per_picture_flag is equal to 1, skip the signalling of sps_num_subpics_minus1 and sps_independent_subpics_flag and infer the values. (JVET-R0060)
   2. When sps_one_slice_per_picture_flag is equal to 1, skip the signalling of subpic_info_present_flag. (JVET-R0118)

Currently we have a flag in the SH to indicate that all pictures in the CLVS have one slice per picture.

We also have a one_slice_per_pic_constraint_flag (and similar for one tile, and one subpicture).

It was asked whether we have parsing dependencies on the general constraint flags. We do not.

In the current semantics of one_slice_per_pic_constraint_flag equal to 1 there is no constraint that the PH be combined with the SH.

AHG Recommendation (cleanup): Add a “general_pic_header_in_slice_header_constraint_flag” (or similar name). (constrained for sensibility with the existing flag, and constrain the combinations with the above syntax elements for sensibility, but do not add further constraints that are not necessary for sensibility). Text will be provided in a revision of JVET-R0118 and software is to be provided by B. D. Choi.

1. Skip the signalling of the 6 PPS flags rpl_info_in_ph_flag, dbf_info_in_ph_flag, sao_info_in_ph_flag, alf_info_in_ph_flag, wp_info_in_ph_flag, qp_delta_info_in_ph_flag under certain condition. (JVET-R0060, JVET-R0113, JVET-R0124)
   1. Skip them when the existing PPS flag no_pic_partition_flag is equal to 1, and infer their values (to be equal to something TBD) under this condition. (JVET-R0113)
   2. Skip them when a new PPS flag pps_one_slice_per_picture_flag is equal to 1, and infer their values to be equal to 0 under this condition. (JVET-R0060)
      1. Also skip the PPS SEs pps_num_subpics_minus1, rect_slice_flag, single_slice_per_subpic_flag, num_slices_in_pic_minus1, and loop_filter_across_slices_enabled_flag (if this flag remains in the PPS) under this condition and infer the values. (JVET-R0060)
   3. Skip them when a new PPS flag all_pic_coding_info_present_in_ph_flag is equal to 1, and infer their values to be equal to 1 under this condition. (JVET-R0124). This can be considered supplemental to item a.

Subitem “a” is straightforward sensibility cleanup. Subitems b and c are basically for coding efficiency savings (saving 5–10 bits) or bypassing unnecessary flexibility in the PPS level.

It was initially agreed to disallow random settings of these flags when the PH is in the SH.

When no_pic_partition_flag is equal to 1 we already don’t send rect_slice_flag, single_slice_per_subpic_flag, num_slices_in_pic_minus1, and loop_filter_across_slices_enabled_flag. And subpic_id_mapping_in_pps_flag prevents sending pps_num_subpics_minus1. One distinction that was pointed out is that the no_pic_partition_flag does not distinguish the case with many tiles and one slice per picture.

It was suggested that the common case would be to send info in the PH unless flexibility is needed in the SH. However, for the RPL and WP, it (currently) matters where the data is signalled; for RPL, it helps to have the slice type, and for WP it helps to have the number of RPL active entries. For the other four, it doesn’t really matter one way or the other. Interacting with this question, there are proposals for signalling the number of active entries in the PH.

AHG recommendation (cleanup): When (pps_)no_pic_partition_flag is equal to 1, skip the 6 PPS flags rpl_info_in_ph_flag, dbf_info_in_ph_flag, sao_info_in_ph_flag, alf_info_in_ph_flag, wp_info_in_ph_flag, qp_delta_info_in_ph_flag and infer them to be equal to 0. Text is provided in JVET-R0113 and software to be provided by its proponent. Any missing sensibility constraints may be added by the editor.

1. When slice headers referring to the PPS have (sh_)picture_header_in_slice_header_flag equal to 1, require alf_info_in_ph_flag to be equal to 1. (JVET-R0200)

Having this be equal to 1 instead of 0 is requested due to wanting the APS information to be early in the NAL unit. However, it was commented that the proposed requirement would be different than discussed for item 2 above.

No action was recommended for this.

1. When slice headers referring to the PPS have (sh_)picture_header_in_slice_header_flag is equal to 1, require rpl_info_in_ph_flag, dbf_info_in_ph_flag, sao_info_in_ph_flag, wp_info_in_ph_flag, qp_delta_info_in_ph_flag to be equal to 1. (JVET-R0202)

AHG recommendation (cleanup): When (sh_)picture_header_in_slice_header_flag is equal to 1, require rpl_info_in_ph_flag, dbf_info_in_ph_flag, sao_info_in_ph_flag, wp_info_in_ph_flag, qp_delta_info_in_ph_flag to be equal to 0.

(Consistency with item 2 suggests the value 0 rather than 1.)

See also item 15).

1. When (sh_)picture_header_in_slice_header_flag is equal to 1, regardless of the value of wp_info_in_ph_flag, pred_weight_table syntax structure is signalled in slice header and not as part of picture_header syntax structure. (JVET-R0220)
   1. Or add a constraint such that (sh_)picture_header_in_slice_header_flag and wp_info_in_ph_flag shall not be both equal to 1 (technically equivalent to requiring the value of wp_info_in_ph_flag to be equal to 0 when slice headers referring to the PPS have picture_header_in_slice_header_flag equal to 1). (JVET-R0220)

This is resolved by the action taken on item 4.

1. Skip the signalling of the SH syntax element (SE) slice_address when the picture contains only one slice and infer its value. (JVET-R0060, JVET-R0104, JVET-R0162, JVET-R0189, JVET-R0202, JVET-R0210)
   1. Skip it when a new PPS flag pps_one_slice_per_picture_flag is equal to 1. (JVET-R0060)
   2. Skip it when the existing SH flag picture_header_in_slice_header_flag is equal to 1. (JVET-R0104, JVET-R0162, JVET-R0189, JVET-R0202, JVET-R0210)

This is already skipped when rect_slice_flag is 1 and the number of slices in the subpicture is 1. However, it is not skipped when the rect_slice_flag is equal to 0 (i.e., raster scan slices) and NumTilesInPic > 1.

It was suggested to only allow (sh_)picture_header_in_slice_header_flag is equal to 1 when (pps_)rect_slice_flag is 1? It was commented that there is some other PPS syntax associated with that case that would need to be signalled. But that other syntax is minimal (just a PPS flag).

AHG recommendation (cleanup): Only allow (sh_)picture_header_in_slice_header_flag is equal to 1 when (pps_)rect_slice_flag is 1.

1. Skip the signalling of the SH SE num_tiles_in_slice_minus1when the picture contains only one slice and infer its value. (JVET-R0060, JVET-R0104, JVET-R0202, JVET-R0210)
   1. Skip it when a new PPS flag pps_one_slice_per_picture_flag is equal to 1. (JVET-R0060)
   2. Skip it when the existing SH flag picture_header_in_slice_header_flag is equal to 1. (JVET-R0104, JVET-R0202, JVET-R0210)

This was resolved by the action recommended for item 6.

1. Skip the SH SE num_tiles_in_slice_minus1 when NumTilesInPic − slice_address is not greater than 1. (JVET-R0210).

JVET-R0248 includes the same change (among other proposed changes).

AHG recommendation (cleanup): Adopt. Text and software are to be provided in a revision of JVET-R0210.

1. Even when skipping of signalling of the SH SE num_tiles_in_slice_minus1 as in the item above is not done, infer num_tiles_in_slice_minus1, when not present, to be equal to NumTilesInPic − 1. (JVET-R0060, JVET-R0104)

This was resolved by the action recommended for item 8.

1. Consider one of the following
   1. Skip the signalling of the SH SE slice_subpic_id when the SH flag picture_header_in_slice_header_flag is equal to 1 infer its value. (JVET-R0189), or
   2. Add a constraint such that when subpic_info_present_flag is equal to 1, the value of picture_header_in_slice_header_flag shall be equal to 0 (technically equivalent to "When picture_header_in_slice_header is equal to 1, the value of subpic_info_present_flag shall be equal to 0." but editorially the constraint should be expressed on picture_header_in_slice_header_flag). (JVET-R0189, JVET-R0202)

AHG recommendation (cleanup): Adopt approach b. (There was some discussion of whether this is already part of the action taken on item 1.)

1. When picture_header_in_slice_header_flag is equal to 1, skip the signalling of the SH SE num_ref_idx_active_override_flag and infer its value to be equal to 0. (JVET-R0202)

This would be overriding the default number of active entries signalled in the PPS content.

It was discussed whether there is really a connection between the number of slices in the picture and the need to be able to change the number of active reference pictures used by that picture. It was commented that these are somewhat different issues.

It was said and confirmed that we have designed the PH syntax structure to be the same regardless of whether the PH is combined with the SH or not.

No action was recommended on this since the coupling seems unnecessary and the issue seems minor.

1. Add a constraint such that when slice headers referring to the SPS contain the PH syntax structure, separate_colour_plane_flag shall be equal to 0. (technically equivalent to "When separate_colour_plane_flag is equal to 1, the value of picture_header_in_slice_header_flag shall be equal to 0.", but editorially the constraint should be expressed on picture_header_in_slice_header_flag). (JVET-R0202)

It was remarked that this is somewhat hypothetical, since no profile supports this.

AHG recommendation (basically editorial cleanup): Adopt.

1. Change the text for determination of the first VCL NAL unit of an AU:
   1. As follows: (JVET-R0163)

If a PH NAL unit is present in a PU, let firstVclNalUnitInPic be the first VCL NAL unit that follows the PH NAL unit; otherwise let firstVclNalUnitInPic be the only one VCL NAL unit in a PU. firstVclNalUnitInPic is the first VCL NAL unit of an AU (and consequently the PU containing the VCL NAL unit is the first PU of the AU) when one or more of the following conditions are true:

• • • The value of nuh_layer_id of the VCL NAL unit is less than the nuh_layer_id of the previous picture in decoding order.
    • The value of ph_pic_order_cnt_lsb of the VCL NAL unit differs from the ph_pic_order_cnt_lsb of the previous picture in decoding order.
    • PicOrderCntVal derived for the VCL NAL unit differs from the PicOrderCntVal of the previous picture in decoding order.
  1. As follows: (JVET-R0124)

A VCL NAL unit is the first VCL NAL unit of an AU (and consequently the PU containing the VCL NAL unit is the first PU of the AU) when the VCL NAL unit is the first VCL NAL unit that follows a PH NAL unit or has picture_header_in_slice_header_flag equal to 1 and one or more of the following conditions are true:

...

This issue had also been discussed in the ticket system for ticket #979, and a couple of approaches were discussed in that system. It was agreed that the current text has a bug. The three ways to fix it are all technically equivalent; the differences are only editorial.

AHG Recommendation (BF / expression of existing intent): Correct the text as described (with the editorial detail delegated to the editor).

Discussion stopped here for AHG Session 1.14 on Monday 13 April at 0915 UTC.

Discussion began here for JVET on 15 April at 1300 UTC (chaired by GJS, JRO, YKW).

1. Mandate the EOS NAL unit for easy detection of the first VCL NAL unit of a coded picture. (JVET-R0163)
   1. Replace the SH flag picture_header_in_slice_header_flag with a variable derived based on the presence of the PH NAL unit. (JVET-R0163)

In the case of combined PH+SH, the PH NAL unit will not be present. At the transition between CLVSs, the proponent indicates that in order to determine that a new CLVS has begun, it may be necessary to parse SEs of the SH to identify the first VCL NAL unit of the new CLVS.

It was asked why we put the PH in the slice NAL unit instead of moving the slice NAL unit payload into the PH (and perhaps renaming the NUT). It was then explained that this would require increasing the number of picture NUTs to be able to convey random access information.

We are currently using the first bit of the SH to indicate whether a PH is combined into it or not.

It was noted that in HEVC it is also necessary to look at the first bit beyond the NAL unit header, where the first_slice_segment_in_pic_flag is located. It was commented that this need has not been a significant problem for HEVC. The proponent pointed out that in VVC this bit is only needed at the transition between CLVSs.

It was commented that, in the RTP payload format for HEVC there is a use of the first_slice_segment_in_pic_flag and it was not considered a problem, whereas a NAL unit may be a large chunk of data in that environment. Systems typically also support timestamps.

It was commented that it is generally necessary to check the PH presence bit anyway if NAL units may be lost.

It was commented that if we take action on this, it should be to require EOS only under the condition when it would be needed.

Other than the proponent, it was considered acceptable for the detection of the new CLVS in this circumstance to involve checking the PH presence bit in the SH, so no action was take on this.

1. Change the semantics of the 6 PPS flags rpl_info_in_ph_flag, dbf_info_in_ph_flag, sao_info_in_ph_flag, alf_info_in_ph_flag, wp_info_in_ph_flag, qp_delta_info_in_ph_flag as follows (JVET-R0251) – removing some uses of “that do not contain a PH syntax structure” and changing a “may” to “shall” (italics only for notes emphasis below):

rpl_info_in_ph_flag equal to 1 specifies that reference picture list information is present in the PH syntax structure and not present in slice headers referring to the PPS that do not contain a PH syntax structure. rpl_info_in_ph_flag equal to 0 specifies that reference picture list information is not present in the PH syntax structure and may be present in slice headers referring to the PPS.

dbf_info_in_ph_flag equal to 1 specifies that deblocking filter information is present in the PH syntax structure and not present in slice headers referring to the PPS that do not contain a PH syntax structure. dbf_info_in_ph_flag equal to 0 specifies that deblocking filter information is not present in the PH syntax structure and may be present in slice headers referring to the PPS. When not present, the value of dbf_info_in_ph_flag is inferred to be equal to 0.

sao_info_in_ph_flag equal to 1 specifies that SAO filter information is present in the PH syntax structure and not present in slice headers referring to the PPS that do not contain a PH syntax structure. sao_info_in_ph_flag equal to 0 specifies that SAO filter information is not present in the PH syntax structure and may be present in slice headers referring to the PPS.

alf_info_in_ph_flag equal to 1 specifies that ALF information is present in the PH syntax structure and not present in slice headers referring to the PPS that do not contain a PH syntax structure. alf_info_in_ph_flag equal to 0 specifies that ALF information is not present in the PH syntax structure and may be present in slice headers referring to the PPS.

wp_info_in_ph_flag equal to 1 specifies that weighted prediction information may be present in the PH syntax structure and not present in slice headers referring to the PPS that do not contain a PH syntax structure. wp_info_in_ph_flag equal to 0 specifies that weighted prediction information is not present in the PH syntax structure and may be present in slice headers referring to the PPS. When not present, the value of wp_info_in_ph_flag is inferred to be equal to 0.

qp_delta_info_in_ph_flag equal to 1 specifies that QP delta information is present in the PH syntax structure and not present in slice headers referring to the PPS that do not contain a PH syntax structure. qp_delta_info_in_ph_flag equal to 0 specifies that QP delta information is not present in the PH syntax structure and shall be present in slice headers referring to the PPS.

Decision (Ed. BF/expression of existing intent): Adopt this change, except using “is” rather than “shall be”.

See also item 4).