JVET-AG0215 AHG9/AHG13: Region-dependent film grain characteristics [P. de Lagrange, E. François, M. Le Pendu, C. Salmon-Legagneur (InterDigital)] [late]
This contribution, while stating that the current FGC SEI message fits its purpose to describe the grain present on scanned film or added for artistic intent on motion pictures, and enables to recover a similar grain by driving synthesis at decoder side at reasonable cost when the grain has been lost in the encoding process for broadcast or streaming purpose, acknowledges that for other uses cases, it may be desirable to describe different film grain in different regions of the picture, for example when the picture is a composite of different sources.
A region-dependent film grain characteristics SEI message is proposed, that includes the existing FGC SEI message syntax, notably independent intensity intervals with model parameters for each interval, and adds elements to describe a list of picture regions, and associate each region with a set of intervals. The commonalities with the FGC SEI message are believed to provide a simple migration path, and simple conversion back to FGC SEI messages for parts of pictures by a network processor, for example in the context of subpictures.
The proposal would also allow different characteristics of film grain in different rectangular regions (not supported by JVET-AG0101 and JVET-AG0140). This includes a common part of the model, and definition of “intensity intervals” to individually control the parameters per region. This would not be the same as controlling the strength by an alpha map, but the effect might be similar.
On the other hand, an alpha map would support more than rectangular regions.
It was commented that all region-based approaches add more complexity and may not be easy to implement. Even frame-based film grain synthesis has not reached a wide deployment so far. The most critical point is sample dependency, where adding another step for alpha blending, or changing the model locally might be a problem. Also on/off requires additional check.
In software, an implementation might be less critical than for the case of dedicated hardware. It was argued that it is up to the decoding device to run it or not (a counter-argument is made that for certain applications the expectation that film grain would be overlaid is important). Use cases may be different, though (more evidence about that would be needed).
It was commented that, as the synthesis related to FGC is computation-intense, the tradeoff with the additional benefit should carefully be considered, and more evidence of the benefit should be provided.
It was requested by proponents to include some approach in TuC. This would require software implementation to also demonstrate the benefit and investigate the complexity impact. Proponents are asked to consider combining their approaches and working on an implementation in software, which would allow more detailed study. See further notes under JVET-AG0328.
JVET-AG0215 AHG9/AHG13: Region-dependent film grain characteristics [P. de Lagrange, E. François, M. Le Pendu, C. Salmon-Legagneur (InterDigital)] [late]
JVET-AG0215 AHG9/AHG13: Region-dependent film grain characteristics [P. de Lagrange, E. François, M. Le Pendu, C. Salmon-Legagneur (InterDigital)] [late]