JVET-AN0257 [AHG11] CPU and GPU time comparison for NNVC models [A. Karabutov, E.Alshina (Huawei), F. Galpin (InterDigital)] [late]

This contribution provides information about the execution time of neural networks across different devices. Two devices were tested: Huawei Mate50 Pro (CPU and NPU) and a high-end PC with an RTX 3090 GPU and CPU AMD Ryzen 9 5950X. LOP6 model from NNVC was used as the model under test. Several frameworks were used for the model inferencing: SADL, PyTorch and SNPE. The results showed that execution time on GPU can be faster than inference by SADL on CPU for the typical size of the processing block up to 15 times.

The current implementation of SADL allows you to control the operations executed by NN elements, but it is not optimized for running on GPUs, on which neural network solutions can demonstrate significant acceleration. To evaluate the complexity of algorithms on real devices, it is proposed to use the algorithm from JVET-AM0073 (including tables that allow the mapping).

Tile size and overlap can also be considered in the calculation, as well as bit depth of integer implementation.

It was commented that guaranteed real-time capability can likely not be deduced by this approach (or at least only for similar hardware architectures).

Further study on other elements of NNVC is recommended, e.g. to find coherence with kMAC/sample, CPU time, etc.

AHG6/AHG12: Enhanced compression beyond VVC capability (69)

Summary and BoG reports

Contributions in this area were discussed during 0905–1250 and during 1435–1535 on Sunday 5 Oct. 2025 (chaired by JRO).