JVET-AK0180 On User Generated Content and non-pristine source videos [J. Samuelsson-Allendes (Sharp)]
This document discusses the topics of User Generated Content (UGC) and Non-Pristine Source (NPS) videos in the context of selecting test material for next-generation video coding standard development. Three different scenarios are described for NPS videos, and it is analysed how conventional methods for subjective and objective quality evaluation applies to these scenarios. Based on the analysis, a few proposals are made for next steps regarding UGC and NPS videos.
The second version of the document adds an example screenshot from one of the UGC test sequences being evaluated at the 37th JVET meeting.
Non-Pristine Source (NPS) is a term that may be used to describe source videos that include visual impairments or degradations that causes them to give a reduced visual experience compared to the “optimal” visual quality. The reason for the impairments can generally be attributed to one or both of the following:
- Non-ideal capturing conditions and capturing equipment. This includes out-of-focus blur, shaking camera, camera noise, rolling shutter skew, etc.
- Pre-compression. This includes for example blockiness, blurriness, over-smoothness, loss of details, motion artefacts, etc.
For the second case, it might happen that a second encoding provides better quality, therefore measuring PSNR versus the pre-compressed “original” is not appropriate.
For scenarios 1 and 2, it would be most appropriate to retain the “documentary style” of UGC, potentially with preprocessing.
As in scenario 3 (upload social platform + transcoding) preprocessing may be used before re-encoding. Something similar might be necessary, or industry might be asked to provide uploaded and preprocessed sequences before re-encoding them, or before/after preprocessing – needs more investigation.
A first step of transcoding already happens on the smartphone, where the content stored after capturing is further compressed for the upload.
“Non pristine” conditions such as low lighting/noisy, shaky, blurred etc. would also be important for industry needs. It would be useful to define a separate class for testing that, potentially with narrower QP range. Content to be selected should be realistic as coming out from camera, and difficult to encode. Also this needs more investigation.
For the current moment, priority should be given to select sequences appropriate for visual testing, and not too simple to encode. This would mostly be “pristine” sequences.