JVET-M0873 BoG report on CE10 related combined and multi-hypothesis prediction contributions [C.-W. Hsu, M. Winken]

This report was reviewed in Track B Wednesday 16 January 1500-1820 (chaired by JRO).

The CE10-related documents were categorized as follows:

• CIIP modifications (12)
• OBMC (2)
• TPM modifications (23)
• LIC (6)
• Other (1)

The BoG met on 13 Jan. 2019 from 6:00pm-10:30pm, 14 Jan. 2019 from 1:00pm-4:00pm, 14 Jan. 2019 from 7:30pm-11:00pm

• Normative changes recommendationed by the BoG:
• Triangular mode:
  1. Adopt syntax change of JVET-M0118: Same as in JVET-M0185, JVET-M0190, M207 (test 1), JVET-M0216 (the first aspect), JVET-M0234 (change corresponding to the result table 7 and 8), JVET-M0317 (section 2.2), JVET-M0328 (test E). This does not signal the triangular prediction mode flag in cases where the combination is not allowed (MMVD, CIIP). Approx. 0.07% rate reduction.
  2. Adopt using only weight group (second weight as in JVET-M0328). The weighting is currently dependent on comparing MVs of the two partitions, loss approx. 0.01% for RA, 0.03% for LB, also simplifies the spec.
  3. Adopt using only one context model for merge_triangle_flag (JVET-M0490). Increases bit rate approx. 0.01% for both RA and LDB, not a real severe complexity issue. Was discussed in Track B, and concluded to better keep it unchanged for now.
  4. Adopt signalling change of triangular merging candidate which does not need LUT (JVET-M0883). This is a combination from various proposals. Does not change merge list construction, simplifies the specification, and gives tiny gain (0.01% both for RA and LB)

Decision: Adopt recommendations 1, 2, and 4 from the list above

• Recommendations for testing in next CE:
• TPM:
  1. Merging candidate list construction (aspects from JVET-M0184, JVET-M0194, JVET-M0233, JVET-M0271, JVET-M0283, JVET-M0286, JVET-M0317, JVET-M0349, JVET-M0399):
     • Re-use general merging candidate list construction as common part
     • Study different variants thereof within CE

It was reported that this comes with basically no loss, and makes the merge not only unified but some proposals also simplify the merge list construction of triangular mode.

It was emphasized in Track B that it might already be an advantage if the first part of merge list construction would be unified, and such a unification should not make the regular merge list derivation more complicated.

Further, the merge list construction of triangular merge list construction is not overly critical, so such modifications shall not penalize compression performance.

From discussion in Track B: This topic was agreed to be included in CE4.

• 1. Signalling (has become obsolete by adoption 4. above, no CE necessary):
     • All proposals use 3 syntax elements for the signalling (split direction and 2 merge indices)
     • Study different variants thereof within CE
• Study combination of triangular prediction and MMVD (within CE about motion vector coding)

From discussion in Track B: As the reported compression gain is probably lower due to the first adoption above, and the current approach has a high impact on encoder runtime, proponents should study the method and whether a fast encoder method is available, make input to the next meeting; no CE was planned.

• CIIP
  1. Study intra/inter weights within a CE (JVET-M0096, JVET-M0454): JVET-M0454 reduces CIIP only with planar mode, and has identical weighting for the entire block (which is signalled). Compression gain 0.1%, but somewhat simplifies by removing unequal weighting, and also does not need special MPM list. JVET-M0096 has 0.1% gain in RA, 0.15% in LB, and uses equal for whole block, and only planar. The part of JVET-M0096 which applies PDPC after blending should be removed (see notes under JVET-M0492 below)
  2. Study MPM generation for CIIP within a Sub-CE of CE3 (for comparison purposes this shall also include methods which don’t use MPM list for CIIP) (JVET-M0183, JVET-M0232, JVET-M0276, JVET-M0296): These target either simplification, harmonization, or improvement of the CIIP MPM coding. Gains are reported up to 0.02%. Compared to the benefits of CE CIIP 1., this does not seem to be competitive, as the latter does not need MPM signalling for CIIP at all, removes the non-uniform blending, and has slightly more gain. This sub-experiment would not have chances for adoption, unless something improves in these regards. Such ideas would better be brought as new proposals by next meeting.
  3. Propagation of intra modes from CIIP blocks shall also be studied within CE (M0294): There is no compression benefit, and though some of the current design my appear “unlogical”, there is no problem with it, and propagating this information may even require more storage. If CE CIIP 1 would be successful, it is not needed anyway.

From discussion in Track B: Establish bullet point 1. as CE.

• LIC (JVET-M0088, JVET-M0115, JVET-M0182, JVET-M0224, JVET-M0450, JVET-M0500)

Given that there will be further action on LIC, to minimize implementation complexity, LIC should be investigated having the following properties:

• 1. Usage of reconstructed intra coded, CIIP and IBC blocks, as this would have impact on complexity and parallel implementation pipelining (test with and without, and further study the impact)
  2. Disabling LIC over CTU boundaries is not necessary
  3. Disable for CIIP blocks
  4. No temporal inheritance of LIC flag
  5. No pruning based on LIC flag in merging candidate list generation
  6. Disable LIC for all subblock modes
  7. The VPDU issue should be solved (e.g., disabling for 128xN or by other method which avoids violating VPDU constraint).
  8. Bi-Prediction issue should be solved (LIC should not have to be applied twice, i.e. for each L0/L1 as done in CE10.5.2). Possible solutions: Do it after bi-predictive combination or restrict LIC to uni-prediction.
• Open issues discussed in Track B:
• CIIP:
  1. JVET-M0492: ask hardware experts about their assessment.

From Track B discussion: It was not possible to conclude. Some concern was raised that in hardware it may not allow re-using existing PDPC logic, as it is usually in pipelining connected to intra prediction, and here it would be connected to inter prediction. Furthermore, the proposal does not provide compression benefit (small loss in RA, small gain in LB, and the reduction of software runtime is low.

• 1. JVET-M0082 raises an issue on appearance of banding within CIIP blocks when horizontal/vertical modes are used, appearing at positions where the weight changes. Though it is not obvious that such such artefacts would still appear in a moving video, it is well noted but would be resolved in case when these modes would be removed, or the non-uniform weighting would disappear. However, there is no benefit (even some small compression loss) when only planar and DC are retained.
• OBMC:
  1. JVET-M0357 has new OBMC results, which is however not directly related to current CE results. It uses uni prediction for the current block, and bi prediction (not integer accuracy, but sub-pel with shorter interpolation filter) for the overlap areas. It provides slightly higher gain than 10.2.2 (0.42% instead of 0.37%), but is not obvious to be less complex (though it has only 5 instead of 6 references to access, it employs additional interpolation, so in terms of signal processing there appears to be more complexity). Following the plenary decision, this is not enough evidence to re-open the OBMC CE.
• JVET-M0424 addresses an issue that might be with interpolation filtering (i.e., requiring 32 bit arithmetic in software)

This issue was discussed on the JVET reflector (see email of F. Bossen) – it was not clear if there is really a problem. Further evidence was considered necessary before any action could be taken. Otherwise, gain by new filters is so small that it would not justify adoption by coding efficiency benefit. This needs further study but was not planned as a topic for a CE.

The current SIMD implementation is with 32 bit.

JVET-M0873 BoG report on CE10 related combined and multi-hypothesis prediction contributions [C.-W. Hsu, M. Winken]

See section 6.10.