Back to Search Document details
41st Meeting: by teleconference, CH, January 2026 2026-01-14 21:14
[AHG11] Stable float convolution for neural network inference
Authors: Honglei Zhang Nokia Francesco Cricri Nokia N. Le Nokia Nannan Zou Nokia B. Koyuncu Nokia Luka Murn Nokia
Abstract
This document proposes a mechanism for using a floating-point data format in convolution operations. It is reported that the proposed mechanism can achieve bit-exact reproducibility across different computation environments and that it completely avoids overflows. The proposed method uses the mantissa bits and sign bit of floating-point numbers in convolution calculations, while keeping bias addition and ReLU/PReLU operations in real floating-point numbers. Float32 is the only data format for hardware performing neural network inference. It is reported that the proposed method, applied to the NNLF in NNVC software, achieves similar performance as the floating-point and int16/int32 format using the SADL library, with about 20% of decoding time when the NNLF is run on a GPU and the rest on a CPU for class B sequences.
Meeting Notes
JVET-AO0178 [AHG11] Stable float convolution for neural network inference [H. Zhang, F. Cricri, N. Le, N. Zou, B. Koyuncu, L. Murn (Nokia)]

This document proposes a mechanism for using a floating-point data format in convolution operations. It is reported that the proposed mechanism can achieve bit-exact reproducibility across different computation environments and that it completely avoids overflows. The proposed method uses the mantissa bits and sign bit of floating-point numbers in convolution calculations, while keeping bias addition and ReLU/PReLU operations in real floating-point numbers. Float32 is the only data format for hardware performing neural network inference. It is reported that the proposed method, applied to the NNLF in NNVC software, achieves similar performance as the floating-point and int16/int32 format using the SADL library, with about 20% of decoding time when the NNLF is run on a GPU and the rest on a CPU for class B sequences.

The authors would like to donate the SW to NNVC, as an optional NN inference mechanism, so JVET experts can study the mechanism further and can choose to report runtimes with neural network-based coding tools running on GPU in addition to CPU.

IEEE 754 specifies float point arithmetic, but summation of more than two summands and overflow are not deterministic.

Inputs of quantized convolutions are quantized: weights are quantized off-line, but temporal buffer values are quantized during inference. During the inference it is needed to compute maximum abs value for input (whole tensor).

Conceptually the same as JVET-AO0148, except:

  • de-scaling factors are float in JVET-AO0178 (power of 2 in JVET-AO0148),
  • de-scaling factor are decided during inference (dynamic quantization), but de-scaling factors in JVET-AO0148 are off-line trained (static quantization).

For some operations (such as max pooling, convolution, multiplication with constant, …) proponent implemented proposed approach in addition to SADL. Some operations such as Sigmoid, Tanh, GeLu, power function, exponential are not supported by proposal (and there is no plan to support). With specific version of CUDA, forced deterministic behaviour, TF32 disabled the results of LOP6 are claimed to be reproducible across devices, some minor losses are observed in test. For some sequence (class A1) abnormal behaviour was observed.

Several experts gave multiple examples of not reproducible operations IEEE 754, because IEEE 754 was not targeting bit-exact reproducibility.

It was confirmed (by NNVC SW coordinators) that NNLF in NNVC is not bit-exact and might have miss match (drift) for video signals out of CTC.

It was commented that run time saving in proposed implementation for GPU tests compared to CPU is lower than expected and was demonstrate in earlier contributions. Proponent confirmed that on-line decision of de-scaling factors (dynamic quantization) is the major source of high increase time.

One expert commented that proper name for technology is operational bit-exact implementation (but not theoretical bit-exact in the worst case).was commented that it would be beneficial to made SW available to JVET members.

Study in EE1 was concluded, as alternative implementation of LOP6 to check speed benefits. Bit-exactness and performance implication was agreed to be the practical way to made this implementation available.

ARCHIVE :
JVET-AO0178_AhG11_StableFloat_v2.docx JVET-AO0178_StableFloat.pptx NNVC-14.1_StableFloat_v2.xlsm
Decisions
Study in EE1 was concluded, as alternative implementation of LOP6 to check speed benefits. Bit-exactness and performance implication was agreed to be the practical way to made this implementation available.
Citation