JVET-AL0156 AHG9/AHG16: Colour calibration for generative face video coding [J. Chen, B. Chen, Y. Ye (Alibaba), S. Yin, S. Wang (CityUHK)]

This contribution proposes to add a colour calibration process on the generated face pictures as an optional post-processing to solve the occasional colour shift issue observed in some generated pictures by the proponent. Three options are provided in the contribution as follows.

• Option 1: add a calibration flag in the GFV SEI message and perform colour calibration on the generated picture based on the distribution parameters derived from the current decoded picture when the calibration flag is on.
• Option 2: add a calibration flag in the GFV SEI message and signal the colour distribution parameters when the flag is on, and perform the colour calibration on the generated pictures based on the distribution parameters signalled.
• Option 3: add an indication flag in GFV SEI message to indicate whether the current generated picture can be used for the colour calibration of the generated picture.

In version 2, Option 3 which has minimum impact on the text was added.

In version 3, based on the comments received in the first round of document review and offline discussion with the HLS expert, the proposed semantics in option 3 was refined.Was already reviewed in HLS. It was already suggested to have a syntax element with meaning that some colour post processing is recommended.

It was asked if the calibration could not also be done by decoder-side postprocessing, without adding signalling per picture, e.g. by histogram comparison and alignment, or colour gamut inconsistencies.

The calibration is only necessary for some sequences where the colour shift occurs.

It was also commented that some of the issues may be caused by the generation in RGB domain which is more unstable than in YUV or other domains.

The issue could probably be resolved by improved synthesis algorithms or additional decoder side processing.

No action for v4 SEI.

AHG17: CfE preparation (15+4)

Contributions in this area were discussed during 1405–1505 and 1525–1725 on Wednesday 26 March 2025 and during 1300–1550 on Friday 28 March 2025 (chaired by JRO).

Some aspects discussed under section 4.4 could also be relevant here.

JVET-AL0156 AHG9/AHG16: Colour calibration for generative face video coding [J. Chen, B. Chen, Y. Ye (Alibaba), S. Yin, S. Wang (CityUHK)]

This contribution proposes to add a colour calibration process on the generated face pictures as an optional post-processing to solve the occasional colour shift issue observed in some generated pictures by the proponent. Three options are provided in the contribution as follows.

Option 1: add a calibration flag in the GFV SEI message and perform colour calibration on the generated picture based on the distribution parameters derived from the current decoded picture when the calibration flag is on.

Option 2: add a calibration flag in the GFV SEI message and signal the colour distribution parameters when the flag is on, and perform the colour calibration on the generated pictures based on the distribution parameters signalled.

Option 3: add an indication flag in GFV SEI message to indicate whether the current generated picture can be used for the colour calibration of the current generated picture.

This contribution proposes a new feature for quality improvement.

It was questioned if this change could be done in an extension rather than in VSEI v4.

It was suggested to just add a note in the v4 semantics that a decoder may choose to apply colour calibration.

Some sample images for option 1 were provided. Video examples can be provided.

No comparison of the three options was provided.

A suggestion was made to clarify in which colour space the colour calibration should be performed.

Some support was expressed for option 3 with some modifications to the language, because it was the simplest option with least change to the specification. Suggested language to consider: ".. the current decoded picture is suitable for colour correction …"

It was suggested to study calibration using the previous reconstructed or source picture.

See the follow-up discussion in section 4.15.

Digitally signed content SEI messages (12)

Contributions in this area were discussed during 0645–1030 and 1300–1630 on Friday 28 March 2025 and 1500-1515 on Monday 31 March 2025 (chaired by J. Boyce).