Experiments with Small Devices #25

ml2cpp, as it is now, allows converting any ML model to a C++ code for inference purposes.

This code is however not yet optimized to run on small devices : these devices may have low CPU speed, low memory, low-power, no FPU, no real OS. ml2cpp Jupyter notebooks were run and inference was tested on a sparc64 or x86-64 server with gigabytes of memory and running full-featured g++ compiler.

We need:

1. get some hardware/micro-controller : STM32, ESP32 and K210 (riscv-64) at least.
2. Check that the generated code can be compiled with the respective gcc versions (C++-17 support, exceptions, RTTI are not always available).
3. check the capacities of these devices (CPU + memory)
4. Check floating point issues : is there an FPU? half-precision floats (float16), bfloat16, ...
5. get an emulator for each device (qemu-???). use it for automated tests.

Priority 1 : Getting at least one of these devices running all these steps (feasibility). choose the one with less constraints (K210 can run a small linux)

The goal is to be able to run the model on the bare metal. No arduino. No MicroPython.

Using STM32, ESP32 or K210 is here only for tuning purposes. Their only added value is to provide a test envoironment so that C++ code will be incrementally adapted in a real-world case.