JVET-AP0170 AHG7: Performance counters for reliable decoding time measurement [S. Hong, Y. Tokumo, T. Ikai (Sharp)]

In standard development, decoder execution time has long been recognized as unreliable due to various noise factors. As part of AHG7's mandate, there is a request to “Develop methodology of more reliable runtime measurement”. JVET-AO0131 proposed perf-cpp usage in reference software. This contribution uses perf as external tool and reported that experimental results confirm a strong correlation between decoder execution time and instruction counts. It is also reported that the overhead enabling performance counters measure is about 6 to 8 % in decoding time. It is recommended to include instruction counts as an option in the reporting template.

It was commented that there are various ways to reduce number of instructions in a software which people might use to reduce the number of instructions. Therefore, it might be undesirable to include this in the reporting template. It may also be difficult to interpret when comparing different software bases.

The information that instruction count gives may also be incomplete, e.g. depending on parallelization. Counting number of cycles might better reflect the runtime.

No agreement on taking action.