JVET-AN0157 AHG17: Valgrind complexity analysis on CfE encoder run time targets [S. Hong, Y. Tokumo, T. Ikai (Sharp)]

In standard development, decoder execution time has long been recognized as unreliable due to various noise factors. As part of AHG7's mandate, there is a request to “Develop methodology of more reliable runtime measurement.”. Experimental results indicate a strong correlation between decoder execution time and instruction references. This contribution asserts that the Valgrind based operation count is more reliable measure. Although the Valgrind decoding is roughly 60 times than usual decoding, this contribution suggests using Valgrind operation count can be beneficial to obtain a more reliable measurement of decoding operation in specific classes.

It was commented that the decoder run time is also highly dependent on memory accesses, cache misses, and in case of multi-core how many instructions can be performed in parallel.

It was asked why for instruction counting not better performance counter would be used, as this does not increase run time.