Local speech-to-text model connection fails on macOS M1 Pro during recording start

[hyper-charlie]

Description

User cannot start meeting note recording with local speech-to-text enabled. Error: "Could not connect to the local speech-to-text model. Please check your settings."

Reproduction Steps

1. Enable local speech-to-text in Settings → Transcription → Local Model
2. Verify model is downloaded and present
3. Start a new meeting note and attempt to begin recording

Environment

• Platform: macOS
• Architecture: M1 Pro (aarch64)
• OS Version: macOS 15.0.1 Sequoia
• App Version: 1.0.13
• Build: 3b53e1b
• Locale: en-GB

Additional Details

• Error occurs consistently when attempting to start recording
• Tested with 2 different local models
• Models are verified as downloaded
• Issue persists after multiple app restarts

Expected Behavior

Recording should start successfully and use the local speech-to-text model for transcription.

[devin-ai-integration]

Codebase Investigation: Local STT Connection Failure on macOS M1 Pro

I explored the relevant code paths for the local STT server lifecycle, connection flow, and frontend error reporting. Here are my findings.

Relevant Files

Backend (Rust)

• plugins/local-stt/src/ext.rs — Core start_server(), get_server_for_model(), health check logic
• plugins/local-stt/src/server/internal2.rs — Cactus-based internal STT server (aarch64 only)
• plugins/local-stt/src/server/external.rs — External sidecar STT server (Argmax/Am models)
• plugins/local-stt/src/server/supervisor.rs — Actor supervision for STT server processes
• plugins/local-stt/src/model.rs — SUPPORTED_MODELS list
• crates/local-model/src/lib.rs — LocalModel enum, is_available_on_current_platform(), #[serde(untagged)] deserialization
• crates/listener-core/src/actors/listener/adapters.rs — WebSocket connection to STT server during recording

Frontend (TypeScript)

• apps/desktop/src/stt/useSTTConnection.ts — Polls getServerForModel every 1s to determine connection status
• apps/desktop/src/stt/useStartListening.ts — Uses conn from useSTTConnection; if null, logs "no_stt_connection" and returns early (no recording starts)
• apps/desktop/src/settings/ai/stt/health.tsx — Produces the error message: "Could not connect to the local speech-to-text model."
• apps/desktop/src/sidebar/toast/registry.tsx — Shows the toast when localSttStatus === "unreachable"
• apps/desktop/src/contexts/notifications.tsx — Polls getServerForModel every 1s to derive localSttStatus
• apps/desktop/src/store/tinybase/store/settings.ts:285-306 — Auto-starts local STT server when current_stt_provider/current_stt_model settings change

Key Finding: Whisper Models Are Unsupported on aarch64 Internal Server

On macOS M1 Pro (aarch64), the start_server() path for Whisper models hits a hard UnsupportedModelType error:

1. LocalModel::Whisper(_) is routed to ServerType::Internal (line 123 of ext.rs)
2. On aarch64, the ServerType::Internal branch only supports LocalModel::Cactus models (lines 160-177 of ext.rs)
3. If the user selected a Whisper model, the match at line 166-168 falls through to _ => return Err(crate::Error::UnsupportedModelType)
4. The whisper-cpp feature is not enabled in the desktop build, so the start_internal_server (whisper-rs based) function is not compiled in

This means: if the user selected a Whisper-type local model on M1 Pro, start_server() will always fail silently (the settings listener catches with .catch(console.error)), and the getServerForModel health check will return None → frontend shows "unreachable".

However, list_supported_models (used in the UI) filters by is_available_on_current_platform(), which returns true for Whisper models on aarch64 Apple Silicon (is_apple_silicon check at line 237 of local-model/src/lib.rs). So Whisper models appear as available in the UI but cannot actually be started on aarch64 when whisper-cpp feature is disabled.

Possible Root Causes (Ranked by Likelihood)

1. Whisper model selected but not startable on aarch64 (Most likely)

• is_available_on_current_platform() returns true for Whisper on Apple Silicon
• But start_server() on aarch64 only handles Cactus models for the internal server path
• The error is silently swallowed by .catch(console.error) in settings listeners
• Fix: Either enable whisper-cpp feature for macOS aarch64 builds, or exclude Whisper models from is_available_on_current_platform() when whisper-cpp is not compiled

2. Supervisor not yet spawned (race condition)

• spawn_stt_supervisor runs in tauri::async_runtime::spawn during plugin setup (lib.rs:83-95)
• If start_server() is called before the supervisor is ready, it returns SupervisorNotFound
• This could happen if settings are restored quickly at app launch

3. Cactus model path resolution issue

• If using a Cactus model, the server looks for the model at cactus_models_dir().join(model.dir_name())
• If the directory exists but is empty or corrupted (download interrupted), is_downloaded returns false but start_server may still attempt to start

Recent Relevant Changes

• 6ac06c35e (Mar 7) — "Improve ws-client retry/error handling and wire connect policy into listener clients (Improve ws-client retry/error handling and wire connect policy into listener clients #4453)" — Changed WebSocket connection logic; added desktop_connect_policy() with 4s timeout, 2 max attempts. This is in the user's build (v1.0.13 tag is 8601f1d9d, before this commit), so this fix isn't in their build.
• 1e7d52b30 (Mar 9) — "fix wrong adapter code for desktop" — Fixed adapter code for Hyprnote/Soniox, also post-v1.0.13.
• d528140ef — "unify local model types in rust" — Refactored LocalModel enum; changes to serde #[serde(untagged)] deserialization could affect how model strings from frontend are parsed in Rust.

Suggested Next Steps

1. Ask the user which specific model(s) they selected — If it's a Whisper model (e.g., QuantizedSmallEn), that confirms root cause i editor #1
2. Check if whisper-cpp feature should be enabled for macOS aarch64 builds — or if Whisper models should be hidden from the UI when the feature is not compiled in
3. Add logging/telemetry to start_server() failures — Currently the error from startServer is caught silently on the frontend; surfacing this would help diagnose similar issues faster
4. Consider an explicit startup health check — The app could verify the server actually started after calling startServer, rather than relying solely on periodic polling

[hyper-charlie]

Additional Testing Details

User has tested this issue with multiple models:

• Char's provided Whisper Large v3 model
• Parakeet v3 multilingual

Error persists across both model options, indicating the issue is not model-specific but rather a connection/service problem on the M1 Pro system.

[hyper-charlie]

Update: After completely quitting and relaunching the app:

• Parakeet worked on first launch, but on second launch gets stuck at "Local transcription is starting up..."
• Whisper Large v3 continues to fail loading

Environment: macOS 15.0.1 Sequoia, M1 (aarch64), App v1.0.13