JVET-AL0221 AHG9: EOI SEI message with luma range adaptation for machine analysis [T. Partanen, M. M. Hannuksela, H. Zhang, A. Aminlou (Nokia)]

Chaired by S. Deshpande on 27 March 2025 at 15:25

The contribution proposes to include the TuC design of luma range adaptation for the encoder optimization information (EOI) SEI message into VSEI v4 with the following changes:

1. Addition of eoi_luma_adaptation_idc, which specifies the type or method for luma range adaptation and for which the following values are specified:
   1. Value 0 specifies that the luma value range has been adapted for display power saving.
   2. Value 1 specifies that luma sample values have been pre-processed by multiplying with lumaRatioMult, which is derived from eoi_ratio_luma_value and eoi_ratio_luma_sign_flag.
2. Gating of the presence of eoi_display_model by eoi_luma_adaptation_idc equal to 0.
3. When eoi_luma_adaptation_idc is equal to 1, addition of eoi_backscale_ratio_value, which indicates whether and by which factor the decoded luma sample values were scaled back in the encoding system when deriving performance metrics (e.g., mAP) used for optimizing the luma range. Decoding systems can use the same back-scaling ratio in their operation.

The contribution provides experiment results as a showcase for demonstrating the usefulness of the proposed signalling for machine analysis tasks. Experiments on the OpenImages dataset on object detection and segmentation task show that the proposed adaptive luma down-scaling and back-scaling predictions achieve -28.6% and -25.0% BD-rate reduction. Version 2 of the contribution adds showcase results on the SFU dataset on object detection task, with -13.2% (LD), -11.7% (RA), and -9.7% (AI) BD-rate reductions.

V2 was discussed.

It was commented that for SFU the back-scaling was computed on the first picture and used for entire sequence.

It was commented that for machine analysis, whether chroma should also be used. The proponent commented that use of chroma has not been tried in this contribution and in JVET-AH0115.

It was asked if this adaptation should be done as bit-depth truncation or more fine grained as proposed here. It was commented by proponents that bit-depth truncation is special case and NN chose more fine grained values and not just 1 bit truncation.

It was asked with the additional changes the purpose description should clarify the additional aspect supported.

It was asked if nonlinear luma adaptation or other ways of doing luma adaptation should be studied and if this is added to VSEI v4, how that would work. It was commented by proponent that the eoi_luma_adaptation_idc can provide further extensibility in future.

Multiple participants commented that this is interesting, but more suitable for TuC at this stage.

Decision: agreed to add to TuC

It was asked by proponent of JVET-AH0115 to consider including that document also in TuC. It was commented that the addition of JVET-AL0221 is a superset of JVET-AH0115 and the additional aspect which is for chroma did not have any results. It was suggested to do further study of JVET-AH0115 which has similar motivation.