FluidInference · Alex-Wengg · Oct 22, 2025 · Oct 12, 2025 · Oct 12, 2025 · Oct 12, 2025
diff --git a/.gitignore b/.gitignore
@@ -1,83 +1,54 @@
-# Prerequisites
-*.d
-
-# Compiled Object files
-*.slo
-*.lo
-*.o
-*.obj
-
-# Precompiled Headers
-*.gch
-*.pch
-
-# Linker files
-*.ilk
-
-# Debugger Files
-*.pdb
-
-# Compiled Dynamic libraries
-*.so
-*.dylib
-*.dll
-
-# Fortran module files
-*.mod
-*.smod
-
-# Compiled Static libraries
-*.lai
-*.la
-*.a
-*.lib
-
-# Executables
-*.exe
-*.out
-*.app
-
-# debug information files
-*.dwo
-
 # Build directories
 build/
 cmake-build-*/
-out/
 
-# Model files and cache
-models/
-cache/
-*.blob
+# Model files (too large for git)
+models/**/*.bin
+models/**/*.blob
 
-# Test audio (can be large)
-test_audio/
 
 # IDE
 .vscode/
 .vs/
-.idea/
-*.swp
-*.swo
-.claude/
+*.suo
+*.user
 
-# Python
-__pycache__/
-*.pyc
-.venv/
-venv/
+# Compiled
+*.o
+*.obj
+*.so
+*.dll
+*.dylib
+*.a
+*.lib
 
-# Node.js
-node_modules/
-dist/
-*.tsbuildinfo
+# Cache
+.cache/
+*.cache
+cache/
 
-# .NET
-bin/
-obj/
-*.user
-*.suo
+# Build artifacts
+**/obj/
+**/bin/Debug/
+**/bin/Release/
 
-# OS
-.DS_Store
-Thumbs.db
+# IDE/Editor specific
+.claude/
+*.sln
+*.vcxproj
+*.vcxproj.filters
+*.vcxproj.user
+
+# Test files
+test_*.wav
+test_*.py
+*_test.cpp
+
+# Large archives
+*.zip
+openvino_*/
+
+# Temporary scripts
+download_*.bat
+download_*.sh
+run_*.bat
diff --git a/AGENTS.md b/AGENTS.md
@@ -0,0 +1,58 @@
+# Repository Guidelines
+
+## Project Structure & Module Organization
+
+- Source in `src/` (core, backends, `models/parakeet/`, pipelines, streaming); public headers in `include/eddy/`.
+- Examples in `examples/cpp/` (`parakeet_cli.cpp`, `benchmark_librispeech.cpp`, optional `whisper_example.cpp`).
+- Scripts in `scripts/` (model fetch); docs in `docs/` (e.g., `docs/Benchmark-Troubleshooting.md`).
+- Assets: sample WAVs in repo root; models live under per-user app data, not in git.
+
+## Build, Test, and Development Commands
+
+- Configure: `cmake -B build -S . -DCMAKE_BUILD_TYPE=Release -DEDDY_ENABLE_OPENVINO=ON`
+- Build tools: `cmake --build build --config Release --target eddy parakeet_cli benchmark_librispeech hf_fetch_models`
+- Run CLI (NPU): `build\examples\cpp\Release\parakeet_cli.exe "<path-to-wav>" --device NPU`
+- Benchmark (NPU): `build\examples\cpp\Release\benchmark_librispeech.exe --max-files 50 --device NPU`
+- Tests (optional): configure with `-DBUILD_TESTING=ON`, then `ctest --test-dir build`
+- Whisper (optional): add `-DEDDY_ENABLE_WHISPER=ON` and set `OpenVINOGenAI_DIR` if not auto-discovered.
+- Python: always use `uv` commands, never use `pip` directly.
+
+## Coding Style & Naming Conventions
+
+- C++20; 2-space indentation; braces on the same line.
+- Types/classes: PascalCase. Functions/variables/files: snake_case (e.g., `parakeet_openvino.cpp`).
+- Keep headers under `include/eddy/...` mirrored by sources under `src/...`.
+- Prefer small, focused functions; avoid inline comments unless clarifying non-obvious logic.
+
+## Testing Guidelines
+
+- Place unit/integration tests under `tests/` (enable with `-DBUILD_TESTING=ON`); name files `<area>_test.cpp`.
+- Manual checks: use `parakeet_cli` and `benchmark_librispeech` with short WAVs and limited file counts.
+- Cover new logic; document any gaps in the PR.
+
+## Commit & Pull Request Guidelines
+
+- Commits: imperative subject with optional scope (e.g., `parakeet: fix encoder port selection`).
+- Keep changes focused; include rationale and before/after behavior.
+- PRs should include summary, reproduction/validation steps, logs or screenshots, target device (CPU/NPU), and linked issues.
+
+## Agent-Specific Instructions (Parakeet/OpenVINO)
+
+- After editing `src/models/parakeet/` or related headers, clear compiled caches but keep downloaded models (Windows: `%LOCALAPPDATA%\eddy\models\parakeet-v2`, preserve `files/`). Quick PowerShell:
+
+  ```powershell
+  $b = "$env:LOCALAPPDATA\eddy\models\parakeet-v2";
+  if (Test-Path $b) {
+    Get-ChildItem $b -File | Remove-Item -Force
+    Get-ChildItem $b -Directory | ? { $_.Name -ne 'files' } | Remove-Item -Recurse -Force
+  }
+  ```
+
+- Rebuild: `cmake --build build --config Release --target eddy parakeet_cli benchmark_librispeech hf_fetch_models`. First run after a cache clear will recompile models and may take minutes.
+
+## Configuration & Models
+
+- OpenVINO env: use `run_bench_npu.bat` to preload. If needed, set `OpenVINO_DIR` (e.g., `C:\Program Files (x86)\Intel\openvino_2025.0.0\runtime\cmake`).
+- GenAI (Whisper): set `OpenVINOGenAI_DIR` when `EDDY_ENABLE_WHISPER=ON`.
+- Download models: Models auto-download on first run via `hf_fetch_models`. Manual download: run `hf_fetch_models.exe` or visit <https://huggingface.co/FluidInference/parakeet-tdt-0.6b-v2-ov> (downloads into `%LOCALAPPDATA%\eddy\models\parakeet-v2\files`).
+- Runtime knobs: `EDDY_OV_PERF`, `EDDY_OV_NUM_REQUESTS`, `EDDY_OV_THREADS`, `EDDY_CONTEXT_FRAMES`, `EDDY_BOUNDARY_SEARCH_FRAMES`, `EDDY_DISABLE_HOLDBACK=1`, `EDDY_DEDUP_PREV_TOKENS` (default 15).
diff --git a/CMakeLists.txt b/CMakeLists.txt
@@ -1,16 +1,34 @@
 cmake_minimum_required(VERSION 3.22)
-project(eddy LANGUAGES CXX)
+project(eddy VERSION 0.1.0 LANGUAGES CXX)
 
 set(CMAKE_CXX_STANDARD 20)
 set(CMAKE_CXX_STANDARD_REQUIRED ON)
+set(CMAKE_CXX_EXTENSIONS OFF)
 set(CMAKE_POSITION_INDEPENDENT_CODE ON)
 
-option(EDDY_ENABLE_OPENVINO "Enable OpenVINO backend" ON)
+# Enable interprocedural optimization (LTO/LTCG) for Release builds
+set(CMAKE_INTERPROCEDURAL_OPTIMIZATION_RELEASE ON)
+
+if(MSVC)
+  # Favor fast math and aggressive inlining for Release
+  # Use flags string to avoid malformed generator expressions on MSBuild
+  string(APPEND CMAKE_CXX_FLAGS_RELEASE " /O2 /Oi /Ot /Ob3 /fp:fast")
+elseif(CMAKE_CXX_COMPILER_ID MATCHES "GNU|Clang")
+  # Equivalent optimization flags for GCC/Clang
+  string(APPEND CMAKE_CXX_FLAGS_RELEASE " -O3 -ffast-math -finline-functions")
+endif()
+
 option(EDDY_BUILD_EXAMPLES "Build examples" ON)
 option(EDDY_BUILD_CSHARP "Build C# bindings" OFF)
+option(EDDY_ENABLE_WHISPER "Enable Whisper (OpenVINO GenAI) support" ON)
 
-if(EDDY_ENABLE_OPENVINO)
-  find_package(OpenVINO REQUIRED COMPONENTS Runtime)
+# Core dependencies
+find_package(OpenVINO REQUIRED COMPONENTS Runtime)
+find_package(SndFile REQUIRED)
+find_package(SampleRate REQUIRED)
+
+# Whisper support (enabled by default, disable with -DEDDY_ENABLE_WHISPER=OFF)
+if(EDDY_ENABLE_WHISPER)
   find_package(OpenVINOGenAI REQUIRED)
 endif()
 
@@ -25,38 +43,65 @@ target_include_directories(eddy
 
 target_sources(eddy
   PRIVATE
+    # Core utilities
     src/core/runtime.cpp
     src/core/model.cpp
+    src/core/app_dir.cpp
+    src/utils/audio_utils.cpp
+    src/utils/ensure_models.cpp
+    src/utils/openvino_utils.cpp
+
+    # OpenVINO backend
     src/backends/openvino_backend.cpp
+
+    # Parakeet TDT v2 implementation
+    src/models/parakeet-v2/parakeet_openvino.cpp
+    src/models/parakeet-v2/parakeet_preprocessor.cpp
+    src/models/parakeet-v2/parakeet_encoder.cpp
+    src/models/parakeet-v2/parakeet_decoder.cpp
+    src/models/parakeet-v2/parakeet_chunking.cpp
+    src/models/parakeet-v2/tokenizer.cpp
 )
 
-if(EDDY_ENABLE_OPENVINO)
-  target_sources(eddy PRIVATE
-    src/pipelines/whisper_pipeline.cpp
-    src/pipelines/audio_utils.cpp
-  )
-  target_compile_definitions(eddy PUBLIC EDDY_WITH_OPENVINO=1)
-  target_link_libraries(eddy PUBLIC openvino::runtime openvino::genai)
+# Link dependencies
+target_compile_definitions(eddy PUBLIC EDDY_WITH_OPENVINO=1)
+target_link_libraries(eddy
+  PUBLIC
+    openvino::runtime
+    SndFile::sndfile
+    SampleRate::samplerate
+)
+
+if(WIN32)
+  target_link_libraries(eddy PRIVATE Shell32)
+endif()
+
+# Whisper support (enabled by default)
+if(EDDY_ENABLE_WHISPER)
+  target_sources(eddy PRIVATE src/pipelines/whisper_pipeline.cpp)
+  target_compile_definitions(eddy PUBLIC EDDY_WITH_OPENVINO_GENAI=1)
+  target_link_libraries(eddy PUBLIC openvino::genai)
 endif()
 
 install(TARGETS eddy EXPORT eddyTargets ARCHIVE DESTINATION lib LIBRARY DESTINATION lib RUNTIME DESTINATION bin)
 install(EXPORT eddyTargets FILE eddyTargets.cmake NAMESPACE eddy:: DESTINATION lib/cmake/eddy)
-install(DIRECTORY include/ DESTINATION include)
+install(DIRECTORY include/
+        DESTINATION include
+        PATTERN "detail" EXCLUDE)
 
 include(CMakePackageConfigHelpers)
 write_basic_package_version_file(${CMAKE_CURRENT_BINARY_DIR}/eddyConfigVersion.cmake
-  VERSION 0.1.0
+  VERSION ${PROJECT_VERSION}
   COMPATIBILITY AnyNewerVersion
 )
-set(EDDY_ENABLE_OPENVINO_CONFIG ${EDDY_ENABLE_OPENVINO})
 configure_file(cmake/eddyConfig.cmake.in ${CMAKE_CURRENT_BINARY_DIR}/eddyConfig.cmake @ONLY)
 install(FILES ${CMAKE_CURRENT_BINARY_DIR}/eddyConfig.cmake ${CMAKE_CURRENT_BINARY_DIR}/eddyConfigVersion.cmake DESTINATION lib/cmake/eddy)
 
 if(EDDY_BUILD_EXAMPLES)
   add_subdirectory(examples)
 endif()
 
-# Build a shared C API shim (eddy_c) that exposes the extern "C" FFI
+# C API shim (always build; Whisper calls are no-ops if EDDY_ENABLE_WHISPER=OFF)
 add_library(eddy_c SHARED src/eddy_c.cpp)
 target_include_directories(eddy_c
   PUBLIC
@@ -65,10 +110,14 @@ target_include_directories(eddy_c
 )
 target_link_libraries(eddy_c PRIVATE eddy)
 target_compile_definitions(eddy_c PRIVATE EDDY_BUILD_SHARED=1)
-
-install(TARGETS eddy_c RUNTIME DESTINATION bin LIBRARY DESTINATION lib ARCHIVE DESTINATION lib)
+install(TARGETS eddy_c
+        EXPORT eddyTargets
+        RUNTIME DESTINATION bin
+        LIBRARY DESTINATION lib
+        ARCHIVE DESTINATION lib)
 
 include(CTest)
-if(BUILD_TESTING)
+# Only add tests if requested and the folder exists
+if(BUILD_TESTING AND EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/tests/CMakeLists.txt")
   add_subdirectory(tests)
 endif()
diff --git a/README.md b/README.md
@@ -2,23 +2,75 @@
 
 eddy is a C++ inference library designed for native runtimes and multi-vendor edge NPUs, exposing a consistent C++ API plus language bindings for app developers (C#; more to follow). The current milestone focuses on the OpenVINO 2025.x backend for the Parakeet-TDT speech model family while we bring additional runtimes online.
 
+## Platform Support
+
+- Supported: Windows and Linux.
+- Not supported: Apple platforms (macOS/iOS). For Apple, use FluidAudio (FA).
+
 ## Repository Layout
+
 - `include/` – public headers for the runtime, backend abstractions, and model bridges.
 - `src/` – backend/runtime implementations and model-specific glue code.
 - `docs/` – design notes and usage guides.
+- `benchmarks/` – Python scripts for LibriSpeech ASR benchmarking (see [benchmarks/README.md](benchmarks/README.md)).
 
-## Building
+## Dependencies
+
+### Required
+- **OpenVINO** (2025.x) - AI inference runtime
+- **libsndfile** - Audio file I/O (WAV, FLAC, OGG, etc.)
+- **libsamplerate** - High-quality audio resampling
+
+### Optional
+- **OpenVINO GenAI** - For Whisper support
+
+### Installing with vcpkg (recommended)
+
+```bash
+# Install vcpkg dependencies (uses vcpkg.json manifest)
+vcpkg install
+
+# Or manually install specific packages
+vcpkg install openvino libsndfile libsamplerate
 ```
+
+## Building
+
+```bash
+# Configure with vcpkg toolchain
+cmake -S . -B build -DCMAKE_TOOLCHAIN_FILE=[path-to-vcpkg]/scripts/buildsystems/vcpkg.cmake
+
+# Or specify OpenVINO manually if not using vcpkg
 cmake -S . -B build -DOpenVINO_DIR=/opt/intel/openvino/runtime/cmake
-cmake --build build
+
+# Build
+cmake --build build --config Release
+```
+
+The build emits the static target `eddy` with all required dependencies.
+
+### Optional Whisper Support
+
+```bash
+cmake -S . -B build -DEDDY_ENABLE_WHISPER=ON -DOpenVINOGenAI_DIR="<path-to-genai-cmake>"
 ```
 
-The build emits the static target `eddy` with OpenVINO linked in when `EDDY_ENABLE_OPENVINO=ON` (default).
+## Models (auto-download on first run)
+
+Models will automatically download from `FluidInference/parakeet-tdt-0.6b-v2-ov` on first use.
+
+Cached at: `%LOCALAPPDATA%\eddy\models\parakeet-v2\files\`
+
+Manual download: Run `hf_fetch_models.exe` or visit <https://huggingface.co/FluidInference/parakeet-tdt-0.6b-v2-ov>
+
+To disable auto-download: set `EDDY_DISABLE_AUTO_FETCH=1`
 
 ## Parakeet OpenVINO Prototype
+
 Refer to `docs/parakeet_openvino.md` for instructions on pulling the exported model from Hugging Face and running a smoke test through the new `OpenVINOParakeet` wrapper.
 
 ## Roadmap Snapshot
+
 - Flesh out Parakeet preprocessing (feature pipeline, tokenizer, decoder).
 - Add telemetry and zero-copy buffers per backend.
 - Introduce unit and integration tests (GoogleTest) with small audio fixtures.