Initial integration of Wasi-nn

[Ahmedounet]

Tflite Inference model Load function implementation following witx conventions of wasi-nn.

[wenyongh]

@Ahmedounet Thanks for uploading the PR, it is really useful! Not sure why you closed the PR, is it due to the CI check failures? It might take some efforts to resolve the CI issues, I am wandering whether we can refactor the PR and upload it by ourselves? We will add you to help review once it is finished.

And of cause, you can resolve the issues and open it again by yourself. There are several suggestions:

• Don't use script install_tensorflow.sh to download the tensorflow source code, instead, we can add FetchContent_Declare in cmake to download it, refer to libc_uvwasi.cmake for how to clone libuv and uvwasi
• For code guideline check, please install clang-format-12 to format the code, refer to https://github.com/bytecodealliance/wasm-micro-runtime/blob/main/ci/coding_guidelines_check.py#L103-L113
• Should add license header in the source files

Thanks.

[tonibofarull]

Hi @wenyongh, Thanks for the feedback! We are working on it in our fork, you will get the PR back once it is stable and tested.

May I take this oportunity to ask some questions?

1. In wasi-nn, the API is expecting some struct parameters, check here: https://github.com/WebAssembly/wasi-nn/blob/main/wasi-nn.wit.md. How do you expect it to be written in WAMR? As far as I know we have to serialize: https://github.com/bytecodealliance/wasm-micro-runtime/blob/main/doc/export_native_api.md#pass-structured-data-or-class-object.
2. Where do we include the tests? Any guideline on this?

Thank you very much!

[wenyongh]

Hi @wenyongh, Thanks for the feedback! We are working on it in our fork, you will get the PR back once it is stable and tested.

May I take this oportunity to ask some questions?

1. In wasi-nn, the API is expecting some struct parameters, check here: https://github.com/WebAssembly/wasi-nn/blob/main/wasi-nn.wit.md. How do you expect it to be written in WAMR? As far as I know we have to serialize: https://github.com/bytecodealliance/wasm-micro-runtime/blob/main/doc/export_native_api.md#pass-structured-data-or-class-object.

Hi, for the structure defintions, we had better align them to the current coding style, for example:

typedef enum TensorType {
    ...
} TensorType;

typedef struct Tensor {
    TensorDimensions dimensions;
    TensorType type;
    TensorData data;
} Tensor;

For the serialization issue, my understanding is there are two isses:
(1) how to access the data structure of wasm world in native, for example, accessing Tensor structure in native api, you can refer to the implementation of libc-wasi wasi_fd_pwrite/pread and wasi_fd_write/read:
https://github.com/bytecodealliance/wasm-micro-runtime/blob/main/core/iwasm/libraries/libc-wasi/libc_wasi_wrapper.c#L412-L429
Normally the layouts in wasm and native are different, in native, we can define both of them:

struct Tensor_app { /* struct in wasm app */
    uint32 dimensions_offset;
    ...;
};
struct Tensor_native { /* struct in native */
    void *dimensions;
};

Then we can validation the app's offset firstly, and then convert it to the pointer (absolute address):

Tensor_app *tensor_app;
Tensor_native *tensor_native;
if (!validate_app_addr(tensor_app->dimensions_offset, sizeof(uint32)) {
  return_error;
}
tensor_native->dimensions = (void *)addr_app_to_native(iovec_app->dimensions_offset);
use tensor_native to call tensorflow API

2. Where do we include the tests? Any guideline on this?

What the tests are? If it is a simple test like unit test, maybe you can put it under tests folder, if it is a workload, had better put it under samples, e.g. samples/wasi-nn.

Thank you very much!