Support for neural network exchange format (NNEF)

[ivan-pi]

The Khronos group has developed a standard data exchange format for neural networks: https://www.khronos.org/nnef. The latest specification is available here: https://registry.khronos.org/NNEF/specs/1.0/nnef-1.0.5.pdf. The purpose of the format is to provide a standard bridge between the various training frameworks and inference engines (e.g. train with Tensorflow, run inference in neural-fortran).

They also provide parsers in C++ and Python and a series of tests: https://github.com/KhronosGroup/NNEF-Tools/tree/master/parser

I'm guessing before committing to implementing and maintaining such a parser in Fortran, it would be cheaper to wrap the existing Khronos C++ parser in Fortran? This could also be brought to life as a module/library separate of neural-fortran.

[milancurcic]

Sorry for the late response--I didn't see this issue until now. Thanks for opening it. I didn't know about NNEF. This could remove the need for framework-specific adapters like that for Keras HDF5.

However, I'm worried about the level of implementation. I searched Keras and PyTorch docs for NNEF and each returned zero results. Is the support for writing NNEF files still in 3rd party libraries?

Related but different is ONNX.

[milancurcic]

In the meantime since this has been opened, I've heard and read a lot about ONNX and nothing about NNEF. I wonder if that means that ONNX is becoming mainstream or a de facto standard for model interexchange.

[ivan-pi]

The NNEF page has added some text explaining the differences

ONNX and NNEF are Complementary
ONNX moves quickly to track authoring framework updates
NNEF provides a stable bridge from training into edge inferencing engines

NNEF	ONNX
Embedded Inferencing Import	Training Interchange
Defined Specification	Open Source Project
Multi-company Governance at Khronos	Initiated by Facebook & Microsoft
Stability for hardware deployment	Software stack flexibility

From what I understand, NNEF aims to be a stable format for (embedded?) inference engines (potentially with hardware support from vendors). I imagine it this way, a software update of the computer vision system in an autonomous car, might constitute simply replacing a NNEF file. The engineers training and tuning the models on the other hand, will exchange ONNX files. The NNEF page says you can export ONNX to NNEF and vice-versa.

So I guess ONNX would be the right one to support first.