Enhance Engine Class Readability and Maintainability (cyrusbehr#72)

* lift and shift inline function to inl files

* revert pathchange for input image

* update readme to include new project structure

Author: thomaskleiven

CMakeLists.txt

@@ -27,7 +27,7 @@ find_package(fmt REQUIRED)
 add_library(tensorrt_cpp_api SHARED
         src/engine.cpp)
 
-target_include_directories(tensorrt_cpp_api PUBLIC ${OpenCV_INCLUDE_DIRS} ${CUDA_INCLUDE_DIRS} ${TensorRT_INCLUDE_DIRS})
+target_include_directories(tensorrt_cpp_api PUBLIC ${OpenCV_INCLUDE_DIRS} ${CUDA_INCLUDE_DIRS} ${TensorRT_INCLUDE_DIRS} include)
 target_link_libraries(tensorrt_cpp_api PUBLIC ${OpenCV_LIBS} ${CUDA_LIBRARIES} ${CMAKE_THREAD_LIBS_INIT} ${TensorRT_LIBRARIES} fmt::fmt)
 
 add_executable(run_inference_benchmark src/main.cpp)

README.md

@@ -134,8 +134,29 @@ Benchmarks run on RTX 3050 Ti Laptop GPU, 11th Gen Intel(R) Core(TM) i9-11900H @
 Wondering how to integrate this library into your project? Or perhaps how to read the outputs of the YoloV8 model to extract meaningful information? 
 If so, check out my two latest projects, [YOLOv8-TensorRT-CPP](https://github.com/cyrusbehr/YOLOv8-TensorRT-CPP) and [YOLOv9-TensorRT-CPP](https://github.com/cyrusbehr/YOLOv9-TensorRT-CPP), which demonstrate how to use the TensorRT C++ API to run YoloV8/9 inference (supports object detection, semantic segmentation, and body pose estimation). They make use of this project in the backend!
 
+### Project Structure
+```sh
+project-root/
+├── include/
+│   ├── engine/
+│   │   ├── EngineRunInference.inl
+│   │   ├── EngineUtilities.inl
+│   │   └── EngineBuildLoadNetwork.inl
+│   ├── util/...
+│   ├── ...
+├── src/
+|   ├── ...
+│   ├── engine.cpp
+│   ├── engine.h
+│   └── main.cpp
+├── CMakeLists.txt
+└── README.md
+```
+
 ### Understanding the Code
-- The bulk of the implementation is in `src/engine.cpp`. I have written lots of comments all throughout the code which should make it easy to understand what is going on. 
+- The bulk of the implementation is located in `include/engine`. I have written lots of comments all throughout the code which should make it easy to understand what is going on. 
+- The inference code is located in `include/engine/EngineRunInference.inl`. 
+- The building and loading of the TensorRT engine file is located in `include/engine/EngineBuildLoadNetwork.inl`.
 - You can also check out my [deep-dive video](https://youtu.be/Z0n5aLmcRHQ) in which I explain every line of code.
 
 ### How to Debug

include/Int8Calibrator.h

@@ -0,0 +1,33 @@
+#pragma once
+#include "NvInfer.h"
+
+// Class used for int8 calibration
+class Int8EntropyCalibrator2 : public nvinfer1::IInt8EntropyCalibrator2 {
+public:
+    Int8EntropyCalibrator2(int32_t batchSize, int32_t inputW, int32_t inputH, const std::string &calibDataDirPath,
+                           const std::string &calibTableName, const std::string &inputBlobName,
+                           const std::array<float, 3> &subVals = {0.f, 0.f, 0.f}, const std::array<float, 3> &divVals = {1.f, 1.f, 1.f},
+                           bool normalize = true, bool readCache = true);
+    virtual ~Int8EntropyCalibrator2();
+    // Abstract base class methods which must be implemented
+    int32_t getBatchSize() const noexcept override;
+    bool getBatch(void *bindings[], char const *names[], int32_t nbBindings) noexcept override;
+    void const *readCalibrationCache(std::size_t &length) noexcept override;
+    void writeCalibrationCache(void const *ptr, std::size_t length) noexcept override;
+
+private:
+    const int32_t m_batchSize;
+    const int32_t m_inputW;
+    const int32_t m_inputH;
+    int32_t m_imgIdx;
+    std::vector<std::string> m_imgPaths;
+    size_t m_inputCount;
+    const std::string m_calibTableName;
+    const std::string m_inputBlobName;
+    const std::array<float, 3> m_subVals;
+    const std::array<float, 3> m_divVals;
+    const bool m_normalize;
+    const bool m_readCache;
+    void *m_deviceInput;
+    std::vector<char> m_calibCache;
+};