Implementing a new hardware backend

[derdiedasoli]

Hi!
I want to try to integrate a different hardware accelerator. Is there some kind of documentation as to which steps are necessary for the integration? If there is, I couldn't find it.

[mwessley]

You are right, there is currently no documentation on the specific steps.
Simple roofline models can be generated by copying the example file in database/models/layer/ncs2-roofline.json and setting the ops and datathroughput values according to you accelerator. (and renaming the file)

The benchmark-based approach is not yet open-sourced (currently sorting out some issues). Are there specific hardware models you are looking for?

[derdiedasoli]

I am looking into integrating more "modern" AI accelerators such as the Hailo-8 NPU or the Axelera Metis AIPU into an inference latency predictor, such as ANNETTE. Since these AI accelerators each use their own backends and require the compilation of each model, the benchmark-based approach would be particularly interesting for this idea. Are there still major issues with the benchmark-based approach, or are you just adding some finishing touches?

[mwessley]

So in general We are working on the open sourcing of the benchmarking process. I havea Hailo-8 NPU lieing around here, maybe I can get to the benchmarking. Axelera is currently sadly currently not on our roadmap.