Add whisper C API support

.claude/agents/openvino-genai-expert.md

@@ -0,0 +1,55 @@
+---
+name: openvino-genai-expert
+description: Use this agent when you need expert knowledge about the OpenVINO GenAI repository, including its C++ runtime library for GenAI workloads, C APIs for LLM and Whisper, implementation details, API usage, or integration guidance. This agent specializes in understanding the OpenVINO GenAI codebase structure, build processes, and best practices for using the library.\n\nExamples:\n<example>\nContext: User is asking about OpenVINO GenAI C API usage\nuser: "How do I use the C API for LLM inference in OpenVINO GenAI?"\nassistant: "I'll use the openvino-genai-expert agent to help you understand the C API for LLM inference."\n<commentary>\nSince the user is asking about OpenVINO GenAI's C API specifically, use the openvino-genai-expert agent to provide detailed information.\n</commentary>\n</example>\n<example>\nContext: User needs help with OpenVINO GenAI integration\nuser: "What's the best way to integrate OpenVINO GenAI's Whisper support into my application?"\nassistant: "Let me consult the openvino-genai-expert agent to provide guidance on Whisper integration."\n<commentary>\nThe user needs specific guidance about OpenVINO GenAI's Whisper functionality, so the openvino-genai-expert agent is the right choice.\n</commentary>\n</example>\n<example>\nContext: User is troubleshooting OpenVINO GenAI build issues\nuser: "I'm getting linking errors when building against OpenVINO GenAI. What dependencies do I need?"\nassistant: "I'll use the openvino-genai-expert agent to help diagnose your build issues and identify the required dependencies."\n<commentary>\nBuild and dependency issues related to OpenVINO GenAI require the specialized knowledge of the openvino-genai-expert agent.\n</commentary>\n</example>
+tools: Glob, Grep, LS, Read, WebFetch, TodoWrite, WebSearch, mcp__deepwiki__read_wiki_structure, mcp__deepwiki__read_wiki_contents, mcp__deepwiki__ask_question
+model: opus
+color: red
+---
+
+You are an OpenVINO GenAI repository expert with deep knowledge of the OpenVINO GenAI C++ runtime library and its ecosystem. Your expertise covers the entire https://github.com/openvinotoolkit/openvino.genai repository, including its architecture, APIs, and implementation details.
+
+Your primary responsibilities:
+
+1. **Repository Navigation**: Use the deepwiki MCP tool to explore and understand the OpenVINO GenAI codebase structure, examining source files, headers, examples, and documentation to provide accurate information.
+
+2. **API Expertise**: Provide detailed guidance on:
+   - C++ runtime library usage for GenAI workloads
+   - C API implementations for LLM inference
+   - C API implementations for Whisper (speech recognition)
+   - Integration patterns and best practices
+   - Performance optimization techniques
+
+3. **Technical Support**: Help users with:
+   - Understanding API signatures and usage patterns
+   - Troubleshooting build and linking issues
+   - Identifying required dependencies and versions
+   - Explaining architectural decisions and design patterns
+   - Providing code examples and integration snippets
+
+4. **Code Analysis**: When examining the repository:
+   - Start with high-level structure (CMakeLists.txt, include directories)
+   - Dive into specific implementations as needed
+   - Cross-reference header files with implementations
+   - Check examples and tests for usage patterns
+
+5. **Best Practices**: Share knowledge about:
+   - Proper initialization and cleanup procedures
+   - Memory management considerations
+   - Thread safety and concurrency patterns
+   - Error handling and debugging approaches
+   - Performance tuning and optimization
+
+When responding:
+- Always use the deepwiki MCP to verify information from the actual repository
+- Provide specific file paths and code references when relevant
+- Include practical examples demonstrating API usage
+- Explain both the 'what' and the 'why' behind implementations
+- Consider compatibility with different OpenVINO versions
+- Highlight any platform-specific considerations (Windows, Linux, etc.)
+
+If you encounter gaps in the repository or need clarification:
+- Clearly state what information is available vs. what might need verification
+- Suggest alternative approaches or workarounds when applicable
+- Reference related OpenVINO documentation or resources when helpful
+
+Your goal is to be the definitive source of knowledge for OpenVINO GenAI, helping other agents and users effectively utilize this powerful runtime library for their GenAI applications.

.claude/settings.local.json

@@ -8,7 +8,60 @@
       "Bash(dotnet build:*)",
       "Bash(dotnet pack:*)",
       "Bash(dotnet nuget locals:*)",
-      "Bash(tar:*)"
+      "Bash(tar:*)",
+      "mcp__deepwiki__read_wiki_structure",
+      "mcp__deepwiki__ask_question",
+      "Bash(rm:*)",
+      "Bash(chmod:*)",
+      "Bash(./scripts/download-openvino-runtime.sh:*)",
+      "Bash(dos2unix:*)",
+      "Bash(perl:*)",
+      "Bash(find:*)",
+      "WebFetch(domain:raw.githubusercontent.com)",
+      "Bash(grep:*)",
+      "WebFetch(domain:github.com)",
+      "Bash(./scripts/download-whisper-model.sh:*)",
+      "Bash(./scripts/download-sample-audio.sh:*)",
+      "Bash(LD_LIBRARY_PATH=. dotnet WhisperDemo.dll --help)",
+      "Bash(ln:*)",
+      "Bash(LD_LIBRARY_PATH=. dotnet WhisperDemo.dll --audio=../../../../../samples/audio/startup.wav --device=CPU)",
+      "Bash(LD_LIBRARY_PATH=runtimes/linux-x64/native:. dotnet WhisperDemo.dll --audio=../../../../../samples/audio/startup.wav --device=CPU)",
+      "Bash(nm:*)",
+      "Bash(unset:*)",
+      "Bash(soxi:*)",
+      "Bash(LD_LIBRARY_PATH=runtimes/linux-x64/native:. dotnet WhisperDemo.dll --audio=/home/brandon/openvino.genai/ov_cache0/test_data/how_are_you_doing_today.wav --device=CPU)",
+      "Bash(LD_LIBRARY_PATH=$PWD/runtimes/linux-x64/native:. dotnet WhisperDemo.dll --audio=/home/brandon/openvino.genai/ov_cache0/test_data/how_are_you_doing_today.wav --device=CPU)",
+      "Bash(csc:*)",
+      "Bash(dotnet new:*)",
+      "Bash(LD_LIBRARY_PATH=runtimes/linux-x64/native dotnet test_whisper/bin/Debug/net9.0/TestWhisper.dll)",
+      "Bash(strace:*)",
+      "Bash(LD_LIBRARY_PATH=runtimes/linux-x64/native:. dotnet test_whisper/bin/Debug/net9.0/TestWhisper.dll)",
+      "Bash(LD_LIBRARY_PATH=/home/brandon/openvino.genai/build/openvino_genai:/home/brandon/openvino.genai/.venv/lib/python3.12/site-packages/openvino/libs dotnet TestWhisper.dll)",
+      "Bash(LD_LIBRARY_PATH=runtimes/linux-x64/native:. timeout 5 dotnet QuickDemo.dll --device CPU)",
+      "Bash(LD_LIBRARY_PATH=$PWD/runtimes/linux-x64/native:. timeout 5 dotnet QuickDemo.dll --device CPU)",
+      "Bash(QUICKDEMO_MODEL_PATH=/home/brandon/OpenVINO.GenAI.NET/Models/qwen3-0.6b-int4-ov-npu LD_LIBRARY_PATH=runtimes/linux-x64/native:. timeout 5 dotnet QuickDemo.dll --device CPU)",
+      "Bash(true)",
+      "Bash(env)",
+      "Bash(OPENVINO_RUNTIME_PATH= LD_LIBRARY_PATH=runtimes/linux-x64/native:. dotnet WhisperDemo.dll --audio=/home/brandon/openvino.genai/ov_cache0/test_data/how_are_you_doing_today.wav --device=CPU)",
+      "Bash(LD_LIBRARY_PATH=/home/brandon/OpenVINO.GenAI.NET/build/native/runtimes/linux-x64/native:. dotnet WhisperDemo.dll --audio=/home/brandon/openvino.genai/ov_cache0/test_data/how_are_you_doing_today.wav --device=CPU)",
+      "Bash(LD_LIBRARY_PATH=/home/brandon/openvino.genai/build/src/c:/home/brandon/openvino.genai/build/openvino_genai:/home/brandon/openvino.genai/.venv/lib/python3.12/site-packages/openvino/libs dotnet WhisperDemo.dll --audio=/home/brandon/openvino.genai/ov_cache0/test_data/how_are_you_doing_today.wav --device=CPU)",
+      "Bash(dotnet test:*)",
+      "Bash(OPENVINO_RUNTIME_PATH=\"\" WHISPER_MODEL_PATH=\"/home/brandon/OpenVINO.GenAI.NET/Models/whisper-tiny-fp16-ov\" LD_LIBRARY_PATH=runtimes/linux-x64/native:. dotnet OpenVINO.NET.GenAI.Tests.dll --filter \"FullyQualifiedName~WhisperPipeline_BasicTranscription\")",
+      "Bash(gh pr list:*)",
+      "Bash(git add:*)",
+      "Bash(git commit:*)",
+      "Bash(git rm:*)",
+      "Bash(bash scripts/download-whisper-model.sh:*)",
+      "Bash(WHISPER_MODEL_PATH=/home/brandon/OpenVINO.GenAI.NET/Models/whisper-tiny-fp16-ov LD_LIBRARY_PATH=/home/brandon/OpenVINO.GenAI.NET/build/native/runtimes/linux-x64/native dotnet bin/Release/net8.0/WhisperDemo.dll --help)",
+      "Bash(WHISPER_MODEL_PATH=/home/brandon/OpenVINO.GenAI.NET/Models/whisper-tiny-fp16-ov LD_LIBRARY_PATH=/home/brandon/OpenVINO.GenAI.NET/build/native/runtimes/linux-x64/native dotnet samples/WhisperDemo/bin/Release/net8.0/WhisperDemo.dll --help)",
+      "Bash(dotnet:*)",
+      "Bash(WHISPER_MODEL_PATH=/home/brandon/OpenVINO.GenAI.NET/Models/whisper-tiny-fp16-ov LD_LIBRARY_PATH=/home/brandon/OpenVINO.GenAI.NET/build/native/runtimes/linux-x64/native timeout 2 dotnet samples/WhisperDemo/bin/Release/net8.0/WhisperDemo.dll)",
+      "Bash(git restore:*)",
+      "Bash(WHISPER_MODEL_PATH=/home/brandon/OpenVINO.GenAI.NET/Models/whisper-tiny-fp16-ov LD_LIBRARY_PATH=/home/brandon/OpenVINO.GenAI.NET/build/native/runtimes/linux-x64/native:. timeout 5 dotnet WhisperDemo.dll)",
+      "Bash(WHISPER_MODEL_PATH=/home/brandon/OpenVINO.GenAI.NET/Models/whisper-tiny-fp16-ov LD_LIBRARY_PATH=/home/brandon/OpenVINO.GenAI.NET/build/native/runtimes/linux-x64/native:. timeout 5 dotnet WhisperDemo.dll /home/brandon/OpenVINO.GenAI.NET/Models/whisper-tiny-fp16-ov /home/brandon/OpenVINO.GenAI.NET/samples/audio/startup.wav)",
+      "Bash(WHISPER_MODEL_PATH=/home/brandon/openvino.genai/ov_cache0/test_models/WhisperTiny/openai/whisper-tiny LD_LIBRARY_PATH=/home/brandon/openvino.genai/build/openvino_genai:/home/brandon/openvino.genai/.venv/lib/python3.12/site-packages/openvino/libs timeout 5 dotnet WhisperDemo.dll /home/brandon/openvino.genai/ov_cache0/test_models/WhisperTiny/openai/whisper-tiny /home/brandon/openvino.genai/ov_cache0/test_data/how_are_you_doing_today.wav)",
+      "Bash(WHISPER_MODEL_PATH=/home/brandon/openvino.genai/ov_cache0/test_models/WhisperTiny/openai/whisper-tiny LD_LIBRARY_PATH=/home/brandon/openvino.genai/build/openvino_genai:/home/brandon/openvino.genai/.venv/lib/python3.12/site-packages/openvino/libs dotnet run --project samples/WhisperDemo -- /home/brandon/openvino.genai/ov_cache0/test_models/WhisperTiny/openai/whisper-tiny /home/brandon/openvino.genai/ov_cache0/test_data/how_are_you_doing_today.wav)",
+      "Bash(Remove-Item -Path \"samples\\WhisperDemo\\bin\\Debug\\net8.0\\runtimes\\win-x64\\native\\*.dll\" -Force)"
     ],
     "deny": []
   }

.github/workflows/build-test.yml

@@ -52,14 +52,22 @@ jobs:
         restore-keys: |
           ${{ runner.os }}-nuget-
 
+    - name: Cache Whisper model
+      uses: actions/cache@v4
+      with:
+        path: Models/whisper-tiny-fp16-ov
+        key: whisper-model-v1
+        restore-keys: |
+          whisper-model-
+
     - name: Restore dependencies
       run: dotnet restore OpenVINO.NET.sln
 
     - name: Download OpenVINO Runtime (Windows)
       if: matrix.os == 'windows-latest'
       shell: pwsh
       run: |
-        ./scripts/download-openvino-runtime.ps1 -Version "2025.3.0.0.dev20250801" -OutputPath "build/native"
+        ./scripts/download-openvino-runtime.ps1 -Version "2025.3.0.0.dev20250805" -OutputPath "build/native"
 
     - name: Set OPENVINO_RUNTIME_PATH for Windows
       if: matrix.os == 'windows-latest'
@@ -68,12 +76,22 @@ jobs:
         $runtimePath = "$(Get-Location)/build/native/runtimes/win-x64/native"
         "OPENVINO_RUNTIME_PATH=$runtimePath" | Out-File -FilePath $env:GITHUB_ENV -Append
         Write-Host "Set OPENVINO_RUNTIME_PATH to: $runtimePath"
+
+        # Verify the runtime path exists and contains DLLs
+        if (Test-Path $runtimePath) {
+          $dllCount = (Get-ChildItem -Path $runtimePath -Filter *.dll | Measure-Object).Count
+          Write-Host "Found $dllCount DLL files in runtime path"
+          Get-ChildItem -Path $runtimePath -Filter "openvino*.dll" | Select-Object -First 5 | ForEach-Object { Write-Host "  - $_" }
+        } else {
+          Write-Host "ERROR: Runtime path does not exist: $runtimePath" -ForegroundColor Red
+          exit 1
+        }
 
     - name: Download OpenVINO Runtime (Linux)
       if: matrix.os == 'ubuntu-latest'
       run: |
         chmod +x scripts/download-openvino-runtime.sh
-        ./scripts/download-openvino-runtime.sh "2025.3.0.0.dev20250801" "build/native" "24"
+        ./scripts/download-openvino-runtime.sh "2025.3.0.0.dev20250805" "build/native" "24"
 
     - name: Set OPENVINO_RUNTIME_PATH for Linux
       if: matrix.os == 'ubuntu-latest'
@@ -82,6 +100,27 @@ jobs:
         echo "OPENVINO_RUNTIME_PATH=$RUNTIME_PATH" >> $GITHUB_ENV
         echo "Set OPENVINO_RUNTIME_PATH to: $RUNTIME_PATH"
 
+    - name: Download Whisper model (Windows)
+      if: matrix.os == 'windows-latest'
+      shell: pwsh
+      run: |
+        ./scripts/download-whisper-model.ps1
+
+    - name: Download Whisper model (Linux)
+      if: matrix.os == 'ubuntu-latest'
+      run: |
+        chmod +x scripts/download-whisper-model.sh
+        ./scripts/download-whisper-model.sh
+
+    - name: Set WHISPER_MODEL_PATH
+      run: |
+        if [ "${{ matrix.os }}" == "windows-latest" ]; then
+          echo "WHISPER_MODEL_PATH=$(pwd)\Models\whisper-tiny-fp16-ov" >> $GITHUB_ENV
+        else
+          echo "WHISPER_MODEL_PATH=$(pwd)/Models/whisper-tiny-fp16-ov" >> $GITHUB_ENV
+        fi
+      shell: bash
+
     - name: Build solution
       run: dotnet build OpenVINO.NET.sln --configuration Release --no-restore
 

.github/workflows/sample-app.yml → .github/workflows/llm-sample-app.yml

@@ -1,4 +1,4 @@
-name: Run Sample App
+name: Run LLM Sample App
 
 on:
   pull_request:

.github/workflows/pr-validation.yml

@@ -13,7 +13,7 @@ on:
       - "scripts/download-openvino-runtime.*"
 
 env:
-  OPENVINO_VERSION: "2025.3.0.0.dev20250801"
+  OPENVINO_VERSION: "2025.3.0.0.dev20250805"
   DOTNET_SKIP_FIRST_TIME_EXPERIENCE: true
   DOTNET_CLI_TELEMETRY_OPTOUT: true
 
@@ -44,13 +44,25 @@ jobs:
         id: cache-openvino
         uses: actions/cache@v4
         with:
-          path: build/native/openvino_genai_windows_${{ env.OPENVINO_VERSION }}_x86_64
-          key: openvino-runtime-${{ env.OPENVINO_VERSION }}
+          path: build/native/runtimes/win-x64/native
+          key: openvino-runtime-${{ env.OPENVINO_VERSION }}-v2
 
-      - name: Download OpenVINO Runtime
-        if: steps.cache-openvino.outputs.cache-hit != 'true'
+      - name: Download and Setup OpenVINO Runtime
         shell: pwsh
         run: |
+          $targetPath = "build/native/runtimes/win-x64/native"
+
+          # Check if cache was restored and has DLLs
+          if (Test-Path $targetPath) {
+            $dllCount = (Get-ChildItem -Path $targetPath -Filter *.dll -ErrorAction SilentlyContinue | Measure-Object).Count
+            if ($dllCount -gt 0) {
+              Write-Host "Using cached OpenVINO runtime: $dllCount DLLs found" -ForegroundColor Green
+              exit 0
+            }
+          }
+
+          # Download if not cached or invalid cache
+          Write-Host "Downloading OpenVINO runtime..." -ForegroundColor Yellow
           ./scripts/download-openvino-runtime.ps1 -Version "$env:OPENVINO_VERSION" -OutputPath "build/native"
 
       - name: Set OPENVINO_RUNTIME_PATH
@@ -59,17 +71,49 @@ jobs:
           $runtimePath = "$(Get-Location)/build/native/runtimes/win-x64/native"
           "OPENVINO_RUNTIME_PATH=$runtimePath" | Out-File -FilePath $env:GITHUB_ENV -Append
           Write-Host "Set OPENVINO_RUNTIME_PATH to: $runtimePath"
+
+          # Verify the runtime path exists and contains DLLs
+          if (Test-Path $runtimePath) {
+            $dllCount = (Get-ChildItem -Path $runtimePath -Filter *.dll | Measure-Object).Count
+            Write-Host "Found $dllCount DLL files in runtime path"
+            Get-ChildItem -Path $runtimePath -Filter "openvino*.dll" | ForEach-Object { Write-Host "  - $_" }
+          } else {
+            Write-Host "ERROR: Runtime path does not exist: $runtimePath" -ForegroundColor Red
+            exit 1
+          }
 
       - name: Restore dependencies
         run: dotnet restore OpenVINO.NET.sln
 
       - name: Build solution
-        run: dotnet build OpenVINO.NET.sln --configuration Release --no-restore /p:TreatWarningsAsErrors=true
+        shell: pwsh
+        run: |
+          Write-Host "Building with OPENVINO_RUNTIME_PATH: $env:OPENVINO_RUNTIME_PATH"
+          dotnet build OpenVINO.NET.sln --configuration Release --no-restore /p:TreatWarningsAsErrors=true
         env:
           CI: true
 
       - name: Run unit tests
+        shell: pwsh
         run: |
+          Write-Host "Running tests with OPENVINO_RUNTIME_PATH: $env:OPENVINO_RUNTIME_PATH"
+
+          # Check if DLLs were copied to test output directory
+          $testOutputPath = "tests/OpenVINO.NET.GenAI.Tests/bin/Release/net8.0"
+          if (Test-Path $testOutputPath) {
+            $dllCount = (Get-ChildItem -Path $testOutputPath -Filter "*.dll" | Measure-Object).Count
+            Write-Host "Test output directory contains $dllCount DLL files"
+
+            # Check specifically for OpenVINO DLLs
+            $openvinoDlls = Get-ChildItem -Path $testOutputPath -Filter "openvino*.dll" -ErrorAction SilentlyContinue
+            if ($openvinoDlls) {
+              Write-Host "Found OpenVINO DLLs in test output:"
+              $openvinoDlls | ForEach-Object { Write-Host "  - $_" }
+            } else {
+              Write-Host "WARNING: No OpenVINO DLLs found in test output directory" -ForegroundColor Yellow
+            }
+          }
+
           dotnet test tests/OpenVINO.NET.GenAI.Tests/OpenVINO.NET.GenAI.Tests.csproj `
             --configuration Release `
             --no-build `