JVET-AQ0156 AHG9: Proposal for a showcase playground [E. Thomas, E. Potetsianakis, E. Alexiou, M.-L. Champel (Xiaomi)] [late]

This contribution presents a web-based playground application for SEI message parsing and showcase. The tool uses the WebCodecs API for hardware-accelerated video decoding in the browser and other Web APIs. There first showcase presented is the decoder picture hash which computes per-frame hashes to verify against the Decoded Picture Hash SEI messages (type 132) embedded in the bitstream. The application is implemented in TypeScript with React and is proposed as a new JVET project.

The WebCodecs API does not expose decoder internals.

Contribution uses a low level decoder rather than the full player. Have to dig into the player to access parameter sets.

The decoded picture hash SEI message was injected into the bitstream created with x.265.

It is suggested to talk to the WebAPI people at W3C.

It is suggested that it might be helpful to have example bitstreams with SEI messages.

WebCodecs may output different formats for YUV. In the proponent’s experiment, the HEVC decoder outputted NV12 ( Y plane + interleaved UV plane)

The JVET TuC repository group Gitlab could potentially host such software. However, this work is not directly related to the TuC, but to HEVC and existing SEI messages.

Github or gitlab can be used to host as well.

A follow-up presentation of this contribution was given in the JVET plenary.

It was asked whether this could also be used for SEI messages that are transporting more than metadata, but additional post processing of the decoder output, or pre-analysis to generate the parameters of the SEI.

It was asked whether the platform that is used is flexible enough to support all standards and all profiles that might be needed in a context of an SEI. More information on that would be used.

It is asserted that the demonstrated tool is useful, but it would need to be considered whether using showcases to make public advertisement of SEI messages (or in case of the proposed tool, a subset of SEI messages) is an effort that should be made by JVET. The current purpose of showcases is for JVET internal use making the potential application of an SEI message more clear.

Developing products out of SEI messages is not the business of JVET, and having a common platform for showcases is not required fo JVET’s purposes.

Experts are encouraged to consider JVET-AQ0156 and investigate whether the benefit of their SEI message could be demonstrated by that environment.

It was agreed to put the software into TuC branch for further consideration