feat: add voxtral-server HTTP transcription server (OpenAI Whisper-compatible)

[kikduck]

Summary

Add a standalone HTTP server (voxtral-server) with an OpenAI Whisper-compatible API for audio transcription, similar to what whisper.cpp and llama.cpp offer.

The model is loaded once at startup and reused across requests. Inference is serialized via std::mutex (voxtral_context is not thread-safe). This avoids the overhead of loading the ~2.8 GB model for each transcription.

New files

• src/server.cpp — HTTP server (~550 lines)
• CMakeLists.txt — New voxtral-server target + FetchContent for cpp-httplib v0.20.0

API Endpoints

Method	Path	Description
GET	/health	Health check → {"status":"ok"}
GET	/v1/models	List loaded model
POST	/v1/audio/transcriptions	Transcribe audio (OpenAI Whisper-compatible)

Transcription endpoint

Accepts two input methods:

• Multipart file upload (curl -F "file=@audio.wav") — standard OpenAI Whisper format
• JSON with base64 ({"audio_base64": "..."}) — convenient for programmatic clients

Returns {"text": "...", "duration": 2.301} (JSON) or plain text.

Additional changes

• CLI --stdin mode (src/main.cpp): reads audio paths from stdin, keeps model loaded between transcriptions, outputs with __VOXTRAL_END__ sentinel. Useful for integration with scripts or Python subprocesses.
• Includes the GPU backend KV cache fix from fix: use backend-agnostic APIs for KV cache on GPU backends #5

Build

mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release   # VOXTRAL_BUILD_SERVER=ON by default
cmake --build . -j$(nproc)
# Produces: voxtral, voxtral-server, voxtral-quantize

To disable: cmake .. -DVOXTRAL_BUILD_SERVER=OFF

Usage

./voxtral-server --model path/to/Q4_K_M.gguf --gpu auto --port 8090

# Test
curl http://localhost:8090/health
curl -X POST http://localhost:8090/v1/audio/transcriptions -F "file=@audio.wav"

Design decisions

• Zero external dependency besides cpp-httplib (header-only, fetched at build time)
• RAII temp files for uploaded audio (auto-cleaned)
• CORS enabled for browser clients
• Signal handling (SIGINT/SIGTERM) for graceful shutdown
• Configurable: host, port, threads, max-tokens, GPU backend, log level