modify

Author: andy6804tw

sklearn-example/onnx-mlir.md renamed to sklearn-example/README.md

@@ -475,7 +475,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": 2,
    "id": "da65a942-c764-4c6b-bbed-813d00ec51b7",
    "metadata": {
     "tags": []
@@ -509,12 +509,59 @@
     "print(f\"ONNX 模型已儲存至 {onnx_file_path}\")"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "cae98f28-7c4b-4b2e-906d-2a4731843cb1",
+   "metadata": {},
+   "source": [
+    "### 1.4 使用ONNX Runtime進行推論測試\n",
+    "我們可以先透過 ONNX Runtime 輸入一筆測試資料檢查推論結果。可以跟稍後 ONNX-MLIR 推論結果進行驗證比較看有沒有數值一至。"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "id": "dfa57338-f3ec-4c89-9225-67b5b5f5e81a",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "[{0: 2.4887262043193914e-05, 1: 0.008561260998249054, 2: 0.9914138317108154}]\n"
+     ]
+    }
+   ],
+   "source": [
+    "import onnxruntime as ort\n",
+    "import numpy as np\n",
+    "\n",
+    "# 加載 ONNX 模型\n",
+    "session = ort.InferenceSession('iris_logistic_regression.onnx')\n",
+    "\n",
+    "# 準備輸入資料\n",
+    "input_name = session.get_inputs()[0].name\n",
+    "input_data = np.array([[6.3, 3.3, 6. , 2.5]], dtype=np.float32)\n",
+    "\n",
+    "# 進行推理\n",
+    "pred_onnx = session.run(None, {input_name: input_data})[1]\n",
+    "\n",
+    "# 輸出預測結果\n",
+    "print(pred_onnx)"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "c6afb752-fe98-435f-a2c0-efe30ed2ee55",
    "metadata": {},
    "source": [
-    "### 1.3 使用 Hummingbird 將模型轉換為 ONNX 格式"
+    "### 1.3 使用 Hummingbird 將模型轉換為 ONNX 格式\n",
+    "onnx-mlir 的設計主要針對深度學習（DL）模型，例如 CNN、RNN 等神經網路，這些模型的特點是以張量（Tensor）為主要數據結構。 Logistic Regression 模型轉換為 ONNX 時，輸出的預測結果默認是 `Sequence<Map>` 類型，用於將分類概率與標籤對應起來。然而，這些類型並不是 onnx-mlir 所支持的張量類型，因此引發錯誤。\n",
+    "\n",
+    "> [1/6] Mon Nov 18 22:05:19 2024 (0s) Importing ONNX Model to MLIR Module from \"iris_logistic_regression.onnx\"\n",
+    "Assertion failed: (elem_type.value_case() == onnx::TypeProto::kTensorType && \"expect tensor inside sequence type\"), function ImportSequenceType, file FrontendDialectTransformer.cpp, line 341.\n",
+    "\n",
+    "一個有效的解決方案是使用 Hummingbird，這是一個專門將傳統機器學習模型轉換為神經網路框架（如 PyTorch）的工具。 Hummingbird 可以將 Logistic Regression、Random Forest 等 scikit-learn 模型轉換為 PyTorch 模型。 轉換後的模型具有純張量輸入輸出結構，非常適合使用 onnx-mlir 編譯。"
    ]
   },
   {
@@ -524,31 +571,26 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# ! pip install onnxruntime==1.19.2 onnx==1.16.1"
+    "# ! pip install onnxruntime==1.19.2 onnx==1.16.1 hummingbird-ml"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": 16,
    "id": "e75efcac-0b6c-4c66-a248-40349081498b",
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Model saved with digest: 1609dbcba26491d9bb02bec919f95901ba2140e5\n"
+      "Model saved with digest: 351bd4be5772877c89c49c8da7aeafa2c4e6b669\n",
+      "Archive:  iris_logistic_regression_torch.zip\n",
+      "  inflating: dist/container.pkl      \n",
+      "  inflating: dist/deploy_model.onnx  \n",
+      "  inflating: dist/model_configuration.txt  \n",
+      "  inflating: dist/model_type.txt     \n"
      ]
-    },
-    {
-     "data": {
-      "text/plain": [
-       "'1609dbcba26491d9bb02bec919f95901ba2140e5'"
-      ]
-     },
-     "execution_count": 4,
-     "metadata": {},
-     "output_type": "execute_result"
     }
    ],
    "source": [
@@ -558,7 +600,42 @@
     "hb_model = convert(onnx_model, 'onnx')\n",
     "\n",
     "# 保存轉換後的 ONNX 模型\n",
-    "hb_model.save('iris_logistic_regression_torch')"
+    "hb_model.save('iris_logistic_regression_torch')\n",
+    "\n",
+    "# 解壓縮資料夾\n",
+    "!unzip -o iris_logistic_regression_torch.zip -d dist"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 19,
+   "id": "bf5121bc-2e91-40f5-ac50-6d7412a0f0a1",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "[[2.4887262e-05 8.5612610e-03 9.9141383e-01]]\n"
+     ]
+    }
+   ],
+   "source": [
+    "import onnxruntime as ort\n",
+    "import numpy as np\n",
+    "\n",
+    "# 加載 ONNX 模型\n",
+    "session = ort.InferenceSession('./dist/deploy_model.onnx')\n",
+    "\n",
+    "# 準備輸入資料\n",
+    "input_name = session.get_inputs()[0].name\n",
+    "input_data = np.array([[6.3, 3.3, 6. , 2.5]], dtype=np.float32)\n",
+    "\n",
+    "# 進行推理\n",
+    "pred_onnx = session.run(None, {input_name: input_data})[1]\n",
+    "\n",
+    "# 輸出預測結果\n",
+    "print(pred_onnx)"
    ]
   },
   {
@@ -578,7 +655,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": 6,
    "id": "37a60184-7b15-47c3-95d7-22340cd7155b",
    "metadata": {
     "tags": []
@@ -588,45 +665,58 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "/bin/bash: line 1: ../onnx-mlir/Release/bin/onnx-mlir: cannot execute binary file: Exec format error\n"
+      "Archive:  iris_logistic_regression_torch.zip\n",
+      "  inflating: dist/container.pkl      \n",
+      "  inflating: dist/deploy_model.onnx  \n",
+      "  inflating: dist/model_configuration.txt  \n",
+      "  inflating: dist/model_type.txt     \n",
+      "[1/6] Mon Nov 18 22:15:35 2024 (0s) Importing ONNX Model to MLIR Module from \"deploy_model.onnx\"\n",
+      "[2/6] Mon Nov 18 22:15:35 2024 (0s) Compiling and Optimizing MLIR Module\n",
+      "[3/6] Mon Nov 18 22:15:35 2024 (0s) Translating MLIR Module to LLVM and Generating LLVM Optimized Bitcode\n",
+      "[4/6] Mon Nov 18 22:15:35 2024 (0s) Generating Object from LLVM Bitcode\n",
+      "[5/6] Mon Nov 18 22:15:35 2024 (0s) Linking and Generating the Output Shared Library\n",
+      "[6/6] Mon Nov 18 22:15:36 2024 (1s) Compilation completed\n"
      ]
     }
    ],
    "source": [
-    "# !unzip -o iris_logistic_regression_torch.zip -d dist\n",
-    "!../onnx-mlir/Release/bin/onnx-mlir --EmitLib iris_logistic_regression.onnx"
+    "!../onnx-mlir/Release/bin/onnx-mlir --EmitLib ./dist/deploy_model.onnx"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "b8fa986e-af75-4cfd-8faa-302836b7d373",
+   "id": "9123da46-ef67-4011-ac1b-8ed8c963ffdb",
    "metadata": {},
    "source": [
-    "### 編寫 C++ 程式以載入並執行模型"
+    "成功轉換共享庫後會產生一個 `deploy_model.so` 檔案。我們可以透過系統指令來觀察這個共享庫相依哪些檔案。"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 34,
-   "id": "036db295-6125-4b96-8dfe-66dbd4038a85",
+   "execution_count": 12,
+   "id": "a0f723de-61e7-40ac-9a07-5bae36b56181",
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "模型输出：2.48873e-05 0.00856126 0.991414 \n"
+      "dist/deploy_model.so:\n",
+      "\t./dist/deploy_model.so (compatibility version 0.0.0, current version 0.0.0)\n",
+      "\t/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1500.65.0)\n",
+      "\t/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.100.3)\n"
      ]
     }
    ],
    "source": [
-    "!g++ --std=c++17 inference.cpp dist/deploy_model.so -o main -I../onnx-mlir/include\n",
-    "!./main"
+    "# mac 使用 otool \n",
+    "# Linux 使用 ldd\n",
+    "!otool -L dist/deploy_model.so"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 33,
+   "execution_count": 9,
    "id": "b062bf8d-9681-4fa4-890b-d723fffa0e08",
    "metadata": {},
    "outputs": [
@@ -646,15 +736,53 @@
     }
    ],
    "source": [
+    "# 檢查共享庫的結構和依賴\n",
     "!ld dist/deploy_model.so"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "b8651b7d-7b38-485c-b178-c866abd5461d",
+   "id": "b8fa986e-af75-4cfd-8faa-302836b7d373",
    "metadata": {},
    "source": [
-    "## TVM"
+    "### 3. 撰寫 C++ 程式進行推論\n",
+    "\n",
+    "- [參考官方文件C Runtime API](https://onnx.ai/onnx-mlir/doxygen_html/OnnxMlirRuntime/index.html)\n",
+    "\n",
+    "### 3.1 撰寫 C++ 程式\n",
+    "\n",
+    "> 請參考 sklearn-example/inference.cpp\n",
+    "\n",
+    "### 3.2 編譯程式 \n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "036db295-6125-4b96-8dfe-66dbd4038a85",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "模型输出：2.48873e-05 0.00856126 0.991414 \n"
+     ]
+    }
+   ],
+   "source": [
+    "!g++ --std=c++17 inference.cpp dist/deploy_model.so -o main -I../onnx-mlir/include\n",
+    "!./main"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b8651b7d-7b38-485c-b178-c866abd5461d",
+   "metadata": {
+    "jp-MarkdownHeadingCollapsed": true
+   },
+   "source": [
+    "# TVM"
    ]
   },
   {
@@ -1257,53 +1385,6 @@
    "source": [
     "lib.export_library(\"c_model.tar\", cc.create_shared, cc=\"g++\")"
    ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "cae98f28-7c4b-4b2e-906d-2a4731843cb1",
-   "metadata": {},
-   "source": [
-    "## 使用ONNX Runtime推論比對結果"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 12,
-   "id": "dfa57338-f3ec-4c89-9225-67b5b5f5e81a",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "[[2.4887262e-05 8.5612610e-03 9.9141383e-01]]\n"
-     ]
-    }
-   ],
-   "source": [
-    "import onnxruntime as ort\n",
-    "\n",
-    "# 加載 ONNX 模型\n",
-    "session = ort.InferenceSession('dist/deploy_model.onnx')\n",
-    "\n",
-    "# 準備輸入資料\n",
-    "input_name = session.get_inputs()[0].name\n",
-    "input_data = np.array([[6.3, 3.3, 6. , 2.5]], dtype=np.float32)\n",
-    "\n",
-    "# 進行推理\n",
-    "pred_onnx = session.run(None, {input_name: input_data})[1]\n",
-    "\n",
-    "# 輸出預測結果\n",
-    "print(pred_onnx)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "f8091b61-1312-4f18-989b-4165203b191f",
-   "metadata": {},
-   "outputs": [],
-   "source": []
   }
  ],
  "metadata": {
@@ -1322,7 +1403,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.18"
+   "version": "3.10.15"
   }
  },
  "nbformat": 4,