Skip to content

ARCHITECTURE.md

Latest commit

 

History

History
418 lines (314 loc) · 45.6 KB

ARCHITECTURE.md

File metadata and controls

418 lines (314 loc) · 45.6 KB
Raw

DevType Technical Architecture & Internals

All documentation · Support

This guide describes implemented subsystem ownership and contracts. Source and tests are the authority; a diagram or a dated audit is not proof of runtime behavior on every host. Start with Development for commands or the documentation index for user guides.

🏛️ High-Level Architecture Overview

DevType is structured into four SwiftPM targets designed for separation of concerns, testability, and crash resilience:

flowchart TD
    Entry[DevTypeApp: main.swift] --> App[DevTypeAppCore: AppDelegate and AppKit UI]
    App --> Engine[ExpanderEngine]
    Input[CGEventTap / TapRunLoopThread] --> Match[TypeAheadBuffer / AbbreviationMatcher]
    Match --> Render[MacroParser / DynamicTemplateEngine]
    Render --> Inject[TextInjectionPipeline]
    App --> Select[Selection capture / AI transform]
    Select --> Inject
    App --> Voice[VoiceSessionCoordinator]
    Voice --> Inject
    Inject --> AX[AXTextWriter]
    Inject --> Paste[HIDKeyPoster / PasteboardBroker]
    AX --> Target[Target application]
    Paste --> Target
    App --> Gate[SecretMenuFlow / BiometricGate]
    Gate --> Secret[SecretStore / AES-GCM archive + Keychain key]
    Gate --> Clipboard[SecretClipboard: deliberate copy]
    App --> Safety[DevTypeSafety: window / KVC / legacy Keychain bridge]
Loading

📦 Target Breakdown

1. ExpanderEngine (Swift Target)

The core library containing matching and business logic, matching algorithms, injection pipelines, macro evaluators, voice dictation subsystem, update checkers, and storage models.

  • Subsystems: Engine (event taps, text injection, type-ahead buffers), Matching (prefix search, abbreviation trie), Macros (Mustache & TextExpander parsing, safe math, date arithmetic), AI (Apple Foundation Models, selection gating, offline Markdown transforms), Voice (smart dictation, multi-engine ASR, durable audio capture, thought-revision correction), Models (snippets, groups, encrypted secret store, usage stats), Permissions (TCC verification, AX capability learning), Sync (TextExpander & Espanso importers, JSON/YAML/CSV exporters), and Updates (distance-aware version ordering, GitHub Releases update checker).
  • Independence: Uses AppKit and system frameworks for platform adapters, while keeping window/controller ownership in DevTypeAppCore. Tests inject or isolate system boundaries; importing the library alone does not make every operation headless.
  • Thread Safety: Concurrency uses locks, serial queues and operation/generation checks. Measured latency and adversarial tests support individual paths; they do not establish a universal timing or race-free guarantee.

2. DevTypeSafety (Objective-C Target)

Small Objective-C entry points contain window-ordering and KVC exceptions and bridge legacy Keychain operations. The AppKit call itself stays inside Objective-C so an exception does not unwind through Swift frames. This is not a general wrapper around Accessibility or pasteboard calls; those callers inspect return values and maintain their own validation. See Sources/DevTypeSafety/DevTypeSafety.m.

3. DevTypeAppCore (Swift Library Target)

The AppKit application layer providing the menu bar status item, snippet editor, inline search palette (⌘/), AI action palette (⌘⌥A), live diff preview, Smart Dictation Liquid Glass HUD, 7-tab Preferences window (⌘,), and onboarding setup wizard.

  • UI ownership: Window and controller work runs on the main thread; delayed callbacks retain explicit lifetime checks.
  • Preferences: PreferencesWindowController owns the window and language rebuilds; Preferences/PreferencesViewController is the tab host (sidebar, lazy panes, selection). Each tab is a PreferencesPaneViewController subclass under Sources/DevTypeAppCore/Preferences/, which carries the shared card, row and list helpers. The Voice tab's card builders and provider operations are split across Preferences/Voice/VoicePreferences+<Card>.swift.

4. DevTypeApp (Executable Target)

The entry point in main.swift delegates to DevTypeAppCore. Keeping the executable separate lets controller tests import the application library.

⚡ Subsystem Deep-Dives

1. Keystroke Interception & Event Tap Engine

Keystroke interception is managed by EventTapEngine and TapRunLoopThread:

sequenceDiagram
    autonumber
    actor User
    participant macOS as macOS Window Server
    participant Tap as CGEventTap (EventTapEngine)
    participant Buffer as TypeAheadBuffer
    participant Target as Target Application

    User->>macOS: Key Down Event
    macOS->>Tap: CGEventTap Callback
    alt Password field active OR App Muted
        Tap-->>macOS: Pass through unaltered
        macOS->>Target: Key Down Event
    else Normal Active Field
        Tap->>Buffer: Push Key Event (UTF-16 Char + Flags)
        Tap->>Buffer: Check Prefix Match
        alt No Match
            Tap-->>macOS: Pass through unaltered
            macOS->>Target: Key Down Event
        else Prefix Match Candidate
            Tap-->>macOS: Swallow Event (Drop)
            Note over Tap: Character erased from target stream
        else Exact Match Triggered
            Tap-->>macOS: Swallow Trigger Key
            Tap->>Buffer: Trigger Text Injection Pipeline
        end
    end
Loading
  • Swallowing Ring Buffer: Intercepted keystrokes are evaluated against active triggers. If an abbreviation prefix is matched, the event tap swallows the keystrokes so they never render in the target document.
  • Dead Keys & IME: The engine tracks input source states, dead keys (accents), and IME composition to prevent breaking non-Latin and accented typing flows.
  • Fail-Closed Security: SecureInputMonitor continuously checks IsSecureEventInputEnabled() and AXContextChecker checks for NSSecureTextField. If active, the event tap immediately yields to ensure password privacy.
  • Secure-input status lifecycle: the monitor polls every 350 ms by default, synchronizes timer ownership and its change gate, and rejects queued callbacks from stopped or replaced generations. The menu-bar UI re-reads Secure Input live before updating the engine flag.
  • Password-field menu presentation: StatusItemPresentation maps the live secure-input signal to a localized key and Copy Secret independently of engine permission/pause diagnostics. StatusItemInteraction routes the primary click directly to InlineSearchPanel.open(mode: .copySecrets) and secondary/Control-clicks to the full menu. StatusItemContext retains the clicked copy affordance during menu tracking; rebuilding happens after NSMenu.popUp returns, outside AppKit open/close callbacks. Explicit palette opens close the previous panel and release its matching suspension before showing the new mode. Search results retain the existing SecretMenuFlow authentication and clipboard handling.

2. Text Injection Pipeline

Once a snippet match is triggered, TextInjectionPipeline coordinates text replacement. Its existing completion guard owns the operation lifetime. A new injection cancels its predecessor; engine stop, watchdog expiry, user input, target changes, and live permission/Secure Input changes invalidate later actions. The target includes the application, AX element and original selection, so moving between fields in one app matters too.

flowchart TD
    Trigger[Snippet Triggered / AI Output / Voice Transcript] --> TargetCheck{Validate Target & Focus}
    TargetCheck -- Focus / Target Invalid --> Abort[Refuse Injection Safely]
    TargetCheck -- Valid --> AXCheck{AX Accessible & Writable?}
    
    AXCheck -- Yes (Primary) --> AXWrite[AXTextWriter: Atomic Range Replacement]
    AXWrite --> CaretAX[Position Caret via AX Attribute]
    CaretAX --> Done([Injection Complete])
    
    AXCheck -- No (Fallback) --> Erase[ErasePlan: Synthesize Backspaces via HID]
    Erase --> PBBackup[PasteboardBroker: Snapshot User Clipboard]
    PBBackup --> PBWrite[Set Expansion Payload on Clipboard]
    PBWrite --> HIDPaste[HIDKeyPoster: Synthesize ⌘V]
    HIDPaste --> Verify{Delivery Verified?}
    Verify --> PBRestore[PasteboardBroker: Restore User Clipboard]
    PBRestore --> CaretHID[Reposition Caret via Arrow Keys]
    CaretHID --> Done
Loading

Callers can supply a session validity check to that same completion guard. Voice delivery uses it so a retired session cannot post from an earlier queued operation. Erase-plan consistency is checked before field reads and again before posting: known text must agree with both the UTF-16 range width and grapheme deletion count, while count-only plans must carry equal counts.

AI replacement, palette insertion and search expansion restore their captured source process through SourceAppDelivery. It uses the engine's bounded focus policy (25 waits at 20 ms), refuses missing/terminated/self targets, stops when another app takes focus, and passes the original-process predicate through the existing injection completion guard. Search captures its source before fill-in/authentication work and obtains a fresh permission snapshot at final delivery. App activation alone never authorizes insertion.

The broker checks operation lifetime, clipboard generation, original ownership count and target immediately before posting, including the Command-to-V modifier gap. HID key bursts check before each pair and release already-posted modifiers. Cursor positioning and trailing keys share the same lifetime. These checks reduce avoidable races; global keyboard events do not form an atomic transaction with another application.

Copy and paste share one HIDKeyPoster command-chord owner. It allocates modifier-release and letter events before posting, rechecks permission and continuation after the modifier gap, and releases Command on refusal. Unmapped keyboard characters return no virtual key; each command chooses its own physical C/V fallback.

The available write paths are:

  1. Accessibility Range Replacement (Primary):

    • AXContextChecker retrieves the focused AXUIElement.
    • AXTextWriter attempts atomic range replacement of the trigger characters using kAXSelectedTextRangeAttribute and kAXValueAttribute.
    • Advantage: Instant, zero clipboard pollution, and non-disruptive to the user's pasteboard.

  2. HID Keystroke & Pasteboard Broker (Fallback):

    • If Accessibility write APIs are blocked (e.g. in some terminal emulators or web apps), the pipeline falls back to synthetic HID events.
    • ErasePlan / EraseExecutor posts synthetic backspaces (kVK_Delete) to erase the typed trigger.
    • PasteboardBroker bounds the prior snapshot, prepares representations before clearing, captures the clear operation's ownership count, and requires every payload/marker write to succeed. Text, image, explicit copy and secret-copy paths share this checked publication policy. Failed publication never schedules a paste. Recovery replaces the owned item set, since writeObjects alone appends to partial contents.
    • Delivery verification requires a relevant transition in the pinned field and range. Existing matching selections, unrelated edits, Unicode case-fold shifts and inconclusive expiry remain unverified; they never authorize automatic replay or corrective text insertion. Only attributed delivery trains latency estimates.
    • Clipboard residency uses injectable monotonic time with bounded stall extensions. Unverified delivery retains the payload for its bounded residency before conditional restoration. Restoration never adopts another writer's count. Snapshot limits remain eight items and four MiB, excluding promised-file representations; this is bounded preservation, not complete clipboard fidelity.

3. Dynamic Template & Macro Engine

Snippet bodies are rendered by two cooperating parsers — MacroParser (TextExpander %...% syntax) and DynamicTemplateEngine (Mustache {{...}} syntax) — sharing one pipeline:

flowchart LR
    Raw[Raw Snippet Template] --> Tokenize[Tokenize: Mustache {{...}} & TextExpander %...%]
    Tokenize --> Parser[AST & Token Parser]
    Parser --> Resolvers{Resolve Dynamic Tokens}
    
    Resolvers --> DateCalc[Date/Time Formatter + Offsets]
    Resolvers --> MathCalc[SafeMathParser: calc: arithmetic]
    Resolvers --> GenCalc[UUID / Random / Counters]
    Resolvers --> NestCalc[Nested Snippets: depth <= 10]
    Resolvers --> ClipCalc[Literal Clipboard Ingestion]
    
    DateCalc --> Assembler[Output Assembler]
    MathCalc --> Assembler
    GenCalc --> Assembler
    NestCalc --> Assembler
    ClipCalc --> Assembler
    
    Assembler --> CaretExtract[Extract {{cursor}} Offset]
    CaretExtract --> FinalText[Rendered Text + Caret Plan]
Loading

  • Mustache Syntax ({{...}}):

    • {{date}}, {{date:yyyy-MM-dd}}: Named presets (16, e.g. us, iso, eu, full) or raw Unicode patterns.
    • {{date:iso:+1d}}, {{date:+1w}}: Date arithmetic through calendar-safe offsets (y M w d h m s units).
    • {{time}}: Localized timestamp.
    • {{clipboard}}: Injects clipboard contents on demand. Clipboard text is preserved as literal data so its template-shaped bytes never become macros.
    • {{calc: <expr>}}: Evaluates safe arithmetic via SafeMathParser (strictly bounded: ≤ 64 chars / 48 tokens, no arbitrary code execution; malformed input is left as literal text).
    • {{cursor}}: Calculates the final caret offset and posts arrow keys to reposition the cursor. First marker wins.
    • {{snippet:<trigger>}}: Recursively resolves nested snippets (depth limited to 10, plus a global budget of 10k substitutions / 2 MB output per pass).
    • {{uuid}}, {{random:1-100}}, {{counter:name}}: Generated values (random specs also accept choice lists and hex:/alnum:/digits:/letters: forms; counters persist across launches).
    • {{upper:…}} / {{lower:…}} / {{title:…}} / {{sentence:…}}: Locale-aware case transforms that nest innermost-first.

  • TextExpander Syntax (%...%):

    • Fill-ins: %filltext:name=X%, %fillarea%, %fillpopup%, optional %fillpart:name=X:default=yes% … %fillpartend% sections.
    • %date:FORMAT% (presets/patterns/offsets), TextExpander date math via %@+1D%.
    • %clipboard, %| (cursor position), %key:enter% / return / tab / esc / space.
    • %snippet:<abbrev>% nesting with the same depth/budget bounds.
    • Generated values %uuid%, %random:1-100%, %counter:name%; case blocks %case:upper% … %caseend%.
    • %% escapes a literal % inside macro bodies; unknown %…% sequences pass through untouched.

Both engines resolve nested snippets in place without disturbing sibling macros, and secret records are excluded from nesting lookups.

MacroRenderContext pins one source and its resolved nested dependencies, date, clipboard, and operation-owned volatile store. UUID/random/counter occurrences evaluate once even across a delayed fill-in render; a separate expansion creates a new context. Counter reservation can leave cancellation gaps, and counter persistence is serialized separately from its reader lock.

MacroDocument carries literal provenance, pending transforms and cursor anchors through both parsers. Generated/external text is never reparsed. Nested transforms follow containment and resolve final UTF-16 anchors after length changes. Rendering has explicit size/work/structure failures consumed before erasure; public adapters drop both text and trailing actions on failure. The expansion lab passes its prepared result into injection instead of rendering volatile values twice.

Event-tap matching uses conservative facts in the immutable match snapshot. Full nested/macro preparation runs on processingQueue after the final claim/gates, and the prepared payload is revalidated before erasure.

Library conflict recovery

SnippetStore.resolveConflicts(keeping:) holds the existing mutation lock and one coordinated write across capture, verified backup, adoption and cleanup. LibraryConflictRecovery validates the chosen document, reads back recovery copies of the selected/current/alternate data, and journals its phase before adoption. It verifies the adopted bytes before removing individual captured NSFileVersion objects. Failed cleanup remains retryable and recovery directories are retained under local application support ConflictRecovery.

The typed outcomes distinguish adoption failure, adoption success, and adoption with cleanup pending. Store cache/digest state changes only after verified adoption; pending cleanup keeps writes blocked. The UI runs this disk work off the main thread and shows the outcome and recovery location. Bounds are 32 alternate versions, 64 MiB per candidate and 256 MiB total recovery data. Temporary native-file-version and fresh-store tests cover the local mechanism; they do not establish live iCloud conflict behavior.

Bounded transcript comparison

TranscriptDiffEngine.compare reports either compared segments or an explicit omitted reason. It bounds each input to 262,144 UTF-16 units and 4,096 tokens, trims common prefixes/suffixes, and limits the LCS matrix to 1,048,576 cells with cancellation checks. When comparison is omitted, the voice HUD keeps the complete cleaned text without presenting a partial diff as a complete comparison.

4. On-Device AI & Offline Transforms

DevType integrates on-device Apple Foundation Models (macOS 26+) for local AI text operations alongside deterministic local transforms:

graph LR
    Highlight[User Selects Text] --> Hotkey[Press ⌘⌥A / Type :fix]
    Hotkey --> SelectionReader[SelectionReader Reads AX Selection]
    SelectionReader --> Gate[SelectionGate Validates Safety]
    Gate --> Transformer[AITextTransformer / Local Transform Engine]
    Transformer --> Preview{Preview or Direct Replace?}
    Preview -- Direct Replace --> Restore[SourceAppDelivery]
    Preview -- Preview --> DiffUI[AIPreviewPanel Diff View]
    DiffUI -- Accept --> Restore
    Restore --> Pipeline[TextInjectionPipeline]
    DiffUI -- Reject --> Discard[Discard Transform]
Loading
  • On-device transforms: AI actions use the Apple Foundation Models API. The OS owns execution and model availability; DevType does not select a particular hardware execution unit. Voice correction has its own provider routing, described below.
  • Interactive Actions: Proofreading, rewriting, paraphrasing, condensing, expanding, tone shifting (friendly/formal), bulletizing, prompt enhancement, code engineering (explain code, docstring generator, fix code, unit tests, regex explanation, SQL queries), conventional git commit messages, JSON conversion, and translation (English ⇄ romanized Telugu/Hindi).
  • Offline Local Markdown Stripper (AIMarkdownStripper): A dedicated deterministic transform (.removeMarkdown) that strips Markdown formatting in microseconds without an AI model, functioning across all supported macOS versions (macOS 14+).
  • Delivery Modes: Each kind declares direct or preview output. Proofread and Remove Markdown default to direct in-place replacement; the rest stream into a diff preview panel (Replace / Copy / Retry / Cancel). Users can override per kind in Preferences → AI.
  • Selection ownership and bounds: Cached text requires a finite nonnegative age and is consumed atomically by its validated generation. Reentrant validation cannot erase a newer publication. UTF-16 range checks avoid overflowing addition. The multi-range fallback accepts at most 64 complete pieces and 200,000 characters including separators; failed pieces or an expired read budget yield no aggregate. Native AX calls retain their per-call messaging timeout.
  • Brokered copy reads: Explicit fallback checks the original process and Secure Input throughout polling and on both sides of lazy clipboard reads. The observed change count must remain stable; a superseding publication is reported as clipboardChanged. Unchanged boards remain failures, and ordinary ⌘A only selects text; the AI action shortcut is ⌘⌥A. Clipboard fallback still cannot prove the publishing process or original field when the host supplies no AX evidence.
  • Layered Output Defense & Markdown Policies: AIPromptLeakGuard extracts instruction clauses corpus-wide and fails closed on injection-boundary verdicts; a per-generation echo stripper removes prompt framing the model repeats back; post-generation checks enforce script policy (e.g. romanized-only translations), line-structure preservation, and Markdown policies (.strip, .stripPreservingLayout, .preserve for code/JSON/SQL).
  • Diagnostics Without Content: AIDiagnosticsStore keeps a bounded ring of failure labels and selection-read evidence — bundle IDs, probe summaries, elapsed ms, character counts only — feeding the -- On-device AI -- section of the diagnostic report. No selection text ever reaches logs.
  • Undo Integration: Transformed text can be reverted with standard ⌘Z or DevType's AI Undo Store (Undo last AI in the Command Palette).
  • Abandon means cancel: AITransformDiscardHandle.discard() cancels the running generation, not only its result. runTransform checks for cancellation at every stream snapshot, chunk and re-roll, and settled() reports when the work has unwound. transformStreaming(after:) queues a replacement behind its predecessor so the single-flight latch never refuses it as .busy. Each partial and completion carries its generation, and the preview panel rejects stale ones before drawing. A cancelled generation is classified .discarded, so it does not count against model reliability in AIDiagnosticsStore.
  • Blank results: the preview normalizes a whitespace-only result to a .decodingFailure so Replace and Copy stay disabled.
  • System deferrals: AITransformError.isSystemDeferral maps memory-pressure refusals from the model service (for example a ModelManagerServices error wrapped in FoundationModels.LanguageModelError) to .deferredBySystem. The localized recovery advice (quit apps, retry) is shared by the preview panel and the dictation HUD (VoiceDictationController.aiFailureMessage). Other AI failures keep the HUD's short generic line.
  • Provider-specific correction transport: correction endpoints are routed through adapter-aware contracts so Ollama and OpenAI-compatible paths are handled explicitly, not by speculative response-shape probing.

5. Independent Secrets and Encrypted Storage

SecretModel owns secret metadata without a trigger, body, image, or AI action. Library schema 3 persists groups and secrets as separate collections in one atomic document. SnippetDocument extracts old secret snippets while retaining their UUIDs and effective group restrictions. SecretManagerViewController and SecretEditorController manage this collection independently; SecretLibraryEdit delegates commits to SnippetStore.mutateGroups and SnippetEditTransaction through a value-free adapter, retaining the existing digest, rollback, and orphan-cleanup guards. Snippet resets and same-named group imports preserve the independent collection. User snippet exports omit it; full library backups and relocation retain both metadata collections.

SnippetStore.loadCurrentSecrets() is the authority for sensitive reads and cleanup: it returns nil while a mutation owns the snapshot or disk no longer matches the cached digest. The gated resolver checks this before and after value retrieval, so delayed authentication cannot use a retired selection. ConsolidatedSecretBackingStore.performExclusiveTransaction extends archive exclusion across editor staging, metadata publication and conditional compensation. Its lock order leaves metadata queries available and shares one monotonic five-second contention budget. Cleanup confirms references after acquiring storage exclusion and carries a separate deferred count into the existing pending-cleanup UI.

Metadata migration never accesses stored values. Unreadable master keys and Keychain fallback values retain the explicit Preferences → Advanced → Repair Secret Storage recovery path. Since the §8.11 redesign, value storage is file-first:

flowchart TD
    User([User Requests Secret / Expansion]) --> Biometric{Touch ID / Password Gate}
    Biometric -- Auth Failed --> Refuse[Refuse Access]
    Biometric -- Authenticated (30s cache) --> MasterKey[Retrieve Master Key from Keychain<br/>com.devtype.masterkey]
    MasterKey --> FileLock[Acquire flock on secrets.enc.lock]
    FileLock --> ReadArchive[Read Encrypted Archive: secrets.enc]
    ReadArchive --> Decrypt[CryptoKit AES-GCM 256 Decryption]
    Decrypt --> ReleaseLock[Release flock]
    ReleaseLock --> Action{Action Type}
    Action -- Copy to Clipboard --> DelibCopy[SecretClipboard: 90s Auto-Clear & Transient Marker]
    Action -- Direct Inject --> DirectInject[TextInjectionPipeline: Atomic Range Write]
Loading
  • Archive: Each value is sealed with CryptoKit AES-GCM (fresh random nonce per seal; nonce + ciphertext + tag as one base64 blob) into a versioned JSON archive (~/Library/Application Support/DevType/secrets.enc, 0600, atomic writes). Bytes the current build cannot vouch for are quarantined aside, never overwritten.
  • Master Key: One 256-bit key generated with SecRandomCopyBytes lives in the login keychain under service com.devtype.app.secret.v2, account com.devtype.masterkey. A successful read is cached in memory. Per-item Keychain values remain when migration cannot safely complete; deletion requires verified encrypted publication, and unreadable keys retain the explicit repair path.
  • Cross-Process Locking: Every archive transaction holds a flock on a validated regular <archive>.lock file, covering source reads, master-key handling, verified publication and tier cleanup. Missing/invalid locks and contention beyond five seconds refuse the transaction with diagnostics; no unlocked transaction runs.
  • Biometric Gate: The existing preference gates access with LocalAuthentication / Touch ID or system password when available. Reuse lasts 30 seconds and is invalidated on app resign-active; each authentication request delivers at most one completion.
  • Concealed Clipboard: Copied secrets carry concealed, transient and auto-generated markers requesting exclusion from compatible clipboard managers. They are auto-purged after 90 seconds only if the pasteboard still holds our write. Other applications must honor those markers for exclusion to work.
  • Structural Redaction: Secret metadata has no value field. Stored values are absent from library JSON, snippet exports and diagnostic reports, and are never prefilled in the editor.

🔒 Concurrency & Threading Model

To ensure zero dropped keystrokes and fluid UI responsiveness:

Component Execution Context Synchronization Primitive
Event Tap Callback Dedicated CFRunLoop (TapRunLoopThread) Non-blocking ring buffer
Abbreviation Matching High-priority background queue Read-Copy-Update / Trie
Snippet Store Background utility queue UnfairLock (OSAllocatedUnfairLock)
Secret Archive Background security queue AES-GCM + cross-process flock
AppKit UI / Panels Main Thread (@MainActor) Swift Concurrency

🛡️ Exception Safety & Resilience

Host accessibility calls can fail when applications terminate or expose incomplete state. AppKit window and KVC operations can also raise Objective-C exceptions. DevType implements:

  1. DevTypeSafety Objective-C Gateways: Window presentation and KVC assignment use exception-catching wrappers. Accessibility call sites inspect AX return codes and validate returned values; the bridge does not wrap every AX or pasteboard call.
  2. Double-Injection Guards: Prevents re-entrant expansion loops.
  3. Automatic Event Tap Recovery: If the macOS window server disables the event tap due to system lag (kCGEventTapDisabledByTimeout), DevType automatically catches the event and re-enables the tap.

🔬 Injection Reliability Kit

Expansion correctness is verified, not assumed. The pipeline carries a set of evidence-producing collaborators:

Component Role
DeliveryVerifier Three-outcome verification (delivered / not delivered / unverifiable) of injected text — "no news" is never treated as success.
PasteboardBroker Owns the snapshot → paste → restore invariant for the HID fallback, with clipboard-residency policy and stall probes so a slow target app cannot eat the user's original clipboard.
TypeAheadBuffer Holds keystrokes typed during an async paste and replays them in order afterwards.
ErasePlan / EraseExecutor Grapheme-correct trigger erasure (AX ranges are UTF-16 units; backspaces remove graphemes), with preconditions verified before any destructive erase. Whitespace folding treats NBSP variants as typed space during field comparisons.
AXWriteCapabilityStore Learns per-app whether AX writes truly work (Electron/Chromium apps can report false success) and routes future expansions down the reliable path.
InjectTimingStore Measures per-app paste delivery times so hold delays adapt instead of using fixed worst-case waits.
InjectTelemetryLog Bounded ring of recent injection outcomes feeding diagnostics — bundle ID, path taken, outcome, refuse reason.

When a typed-trigger erase disagrees with AXValue, EraseExecutor can corroborate the exact UTF-16 erase window through the existing SelectionReader range API (AXStringForRange, then its attributed form). Recovery requires a collapsed selection that remains unchanged across the read, matching text and unit counts, and a range no larger than 32,768 UTF-16 units. Conflicting views permit only HID erasure; missing or disagreeing range evidence still refuses. Undo, voice, and erasure after a possible write retain their stricter guards. Refusals include a content-free rangeProbe status.

Backspace expansion undo is prepared by TextInjectionPipeline and delivered through the same serialized injection operation as normal text. Its single-consumption record binds the original PID and focused element, while input revision and undo generation revoke stale continuations. EraseExecutor.Intent.undo forces strict verification, preserves active selections, and keeps the unreadable-field policy through the final asynchronous erase. Preserving text typed afterward requires the recorded insertion position and a matching bounded field window; the nearest matching substring cannot establish ownership when text repeats. The DevTypeExpansionUndoEnabled preference defaults to enabled; changing it consumes existing records and revokes queued undo. Undo delivery retains postedUnverified until delivery evidence exists and cannot register an inverse undo record.

Type-ahead retention has both a grapheme capacity and a 4,096 UTF-16-unit payload ceiling, so a burst of combining marks cannot bypass the storage bound. Overflow flushes admitted text exactly once and passes the current event through. Nonfinite hold intervals use the finite default; negative layout-buffer capacities behave as zero capacity.

Held expansion: When two triggers share a prefix (`slm vs `slmabout), HeldExpansionCoordinator + TriggerPrefixIndex hold the match for a debounce window so longer triggers stay reachable, serialized across the tap thread, main thread, and timers.

Synthetic-event marker: Reinjected keys carry a magic userData tag (0x534E_4950, "SNIP") so the event tap never mistakes its own output for typing — closing the re-expansion loop class of bugs.

🔍 Search & Ranking

The Command Palette and snippet search are offline-first:

  • SnippetSearch folds diacritics/case/width with a per-character origin table so highlight ranges stay correct across grapheme clusters; fields searched: trigger, title, tags, group, content.
  • UsageStatsStore (snippets, UUID-keyed) and CommandUsageStatsStore (palette commands, string-keyed) compute a saturating frequency boost (~log₂ of use count) plus a recency kicker within 7 days, capped at 12 points — deliberately bounded so a hot item cannot outrank an exact trigger match. Ephemeral math, date, and routed rows are excluded from command usage, and macro palette entries use the same bounded usage concepts.
  • Optional Stage-1 semantic boost uses offline NLEmbedding word vectors to reorder results; it never blocks rendering. The model-backed Stage-2 router (PaletteToolRouter) is wired behind the Semantic Search Routing preference and ships off by default. When enabled and a Foundation Models engine is available, it debounces for 250 ms, calls only DevType-owned tools, rejects prose/oversized results, and drops answers for stale queries.
  • Query-level result caches are keyed by library fingerprint, usage-stats revision, UI language, and clipboard state, then invalidated wholesale on changes.

🎙️ Smart Dictation & Audio Journaling (Jot Architecture)

The Voice Dictation subsystem provides speech-to-text with semantic formatting, local-first and provider-neutral: recognition and correction are separate capabilities, resolved per session rather than hard-wired to one model.

  • Selectable engines (TranscriptionEngine): .appleSpeech uses on-device Apple recognition and deterministic cleanup; .localLLM adds Apple Foundation Models or loopback correction; .whisperLocal uses the configured loopback Whisper server; .gemini uses explicitly consented cloud transcription. VoicePreferences.effectiveEngine preserves the selected engine even when its key is missing. Missing cloud prerequisites fail before recording and never silently switch providers.
  • Readiness and assets: VoiceSessionSnapshotFactory prefers SpeechAnalyzer on macOS 26+ and the legacy Apple recognizer below it. SpeechProviderRegistry probes the preferred provider and can use the legacy floor only after it independently reports ready. SpeechAnalyzer assets install through an explicit Preferences action. Whisper setup verifies its model artifact independently from endpoint readiness.
  • Immutable Session Snapshot (VoiceSessionSnapshotFactory): at the moment dictation starts, the engine choice is frozen into a snapshot carrying the speech provider, correction provider, privacy route, correction policy, vocabulary, target lease, and timeout — so changing a preference mid-dictation cannot retarget a running session, and the manifest on disk records exactly what produced each transcript.
  • Enforced Privacy Routes (PrivacyRoute): each engine implies onDeviceOnly, localNetworkOnly, or cloudPermitted; SpeechProviderRegistry filters and resolves providers against the session's route rather than trusting the call site.
  • Segment admission (SpeechSegment / VoiceSessionReducer): live and batch streams ignore older revisions, conflicting equal-revision replays and final-to-volatile downgrades. A same-revision volatile-to-final promotion remains supported. Each result has a 128 KiB text/alternatives budget, at most 32 alternatives and a 256-byte ID; timestamps/confidence must be valid. Retained session data is bounded to 1 MiB, with 512 live or 4,096 batch segments. A budget violation fails visibly and cleans up, preserving existing artifacts. Completion repair considers all accepted batch segments.
  • Serial delivery (VoiceInsertionService): one live edit settles before the next diff is computed; revisions arriving meanwhile update the bounded assembler. Final delivery joins pending live work and maps the real injection outcome into a receipt. Failed/refused operations cannot become an inserted outcome. Late callbacks cannot mutate a replacement session, and an uncertain partial post stops further automatic writes. Cancellation with a pending proposal preserves text instead of guessing at a compensating erase. Main-queue delivery reservations are separately bounded to 1 MiB and 4,096 callbacks, and check the captured session generation when dispatched.
  • Audio Capture & Crash Journaling (DurableVoiceCapture / CAFSessionWriter): 16kHz mono 16-bit PCM written continuously to capture.caf in a per-session directory under ~/Library/Application Support/DevType/VoiceSessions/, recoverable by VoiceRecoveryService after a crash mid-recording.
  • Thought-Revision & Smart Polish (CorrectionPipeline): Inspired by Google Gemini's Jot, resolves spoken self-corrections mid-sentence, strips filler words/hesitations, replaces custom vocabulary jargon, and adapts tone (natural, email, chat, code, verbatim) — with CorrectionValidator holding the result to the session's CorrectionPolicy so a rewriting model cannot exceed what the user allowed.
  • Local endpoint guardrails (LocalEndpointSecurity & LocalCorrectionEndpointRoute): local services are validated as loopback-only, refuse credentials in URL, reject redirects, and use explicit budgets by operation (readiness, catalog, inference). Local correction routing distinguishes Ollama chat/generate versus OpenAI completion contracts before decoding provider output.
  • Voice Dictation HUD (VoiceHUDPanel): Floating non-activating AppKit HUD. On macOS 26+ uses runtime NSGlassEffectView Liquid Glass with an inset organic blob mask; older systems fall back to NSVisualEffectView. Expands with live STT transcript and meters mic RMS. Blob geometry is DevType-owned (LiquidBlobGeometry) — Apple does not provide Liquid Glass shader/orb assets for third-party porting.

🧪 Privacy-Safe Diagnostics

Three infrastructure pieces now define the boundary for diagnostics and failure observability:

  • DiagnosticPrivacy: strips raw user content from reports by emitting bounded shape metadata (lengths, format counts, bounded hash) for values like triggers and selections.
  • BoundedUTF8Tail: enforces explicit retention ceilings for textual evidence, with deterministic oversize/drop accounting.
  • VoiceTerminalDiagnostic: persists typed terminal outcomes (failed, cancelled, superseded) with stage/provider/locality metadata so recovery policy stays explicit and machine-readable.
  • ErrorGraph: a bounded, cycle-safe breadth-first walk over an error and the errors it wraps (NSUnderlyingErrorKey, NSMultipleUnderlyingErrorsKey). Apple frameworks often report the real cause several levels below the error a caller catches, so classification and logging read the whole tree rather than the outer code. Depth (8) and node (64) caps guarantee termination, and a truncated result means an empty lookup does not prove absence.
  • DevTypeLog.errorMetadata: records the cause chain as causes=domain:code,…. Framework domains appear as-is; other domains are replaced by a salted fingerprint so file paths and request contents cannot leak through error metadata.
  • InjectRefusalKind: the typed classification of an InjectOutcome.refused. classify(reason:path:) owns the mapping to both the kind and the localized sentence, so they cannot diverge. PermissionRecoveryController shows guidance from the kind, and warrantsEngineRestart decides whether the "Restart Engine" menu action is offered.

🔔 Updates & Release Verification Subsystem

DevType contains a dedicated, privacy-conscious update checking module (Sources/ExpanderEngine/Updates/):

  • Zero-Telemetry Version Check (UpdateChecker): Queries the public GitHub Releases API for bharathvbcr/DevType. Carries no device identifiers, cookies, versions, or user analytics.
  • Strict Distance-Aware Version Parsing (AppVersion): Distinguishes git describe distance suffixes (e.g. v0.1.2-3-gabc1234 is ahead of v0.1.2, not a pre-release).
  • Navigation Safety: Release URLs are strictly constructed locally for https://github.com/bharathvbcr/DevType/releases/tag/v... using strict alphanumeric and semantic version checks to prevent arbitrary URI scheme or host traversal.
  • Fail-Closed Outcome Typing (UpdateCheckOutcome): Distinguishes between .upToDate, .updateAvailable, .failed, and .undeterminedLocalVersion, ensuring network outages never report a false "up to date" result.

Shared runtime and UI owners

  • DebouncedSidecarWriter owns usage-sidecar scheduling, serialized atomic writes, retry scheduling and termination flush. The snippet and command usage stores implement SidecarPayloadSource to supply their pending bytes and re-arm failed writes.

  • SupportDirectory resolves the application-support location with a temporary-directory fallback; FilePermissions owns file mode application and atomic publication for secret archives and voice session records. Its unique staging file is owner-only before the first content write and replaces the destination with one rename. Voice record limits match the recovery reader. Saturating arithmetic is shared by timing, capability and usage counters where those callers require the same overflow behavior.

  • A loaded XCTest bundle redirects default support paths to one unique temporary directory per test process, including release-mode tests. Shared voice diagnostics, recovery files and AX learning sidecars therefore use test storage rather than the installed app's support directory. Explicit fixture paths remain under the fixture's control; this does not redirect Keychain or arbitrary external paths.

  • SingleFlightLatch is shared by AI operations that refuse overlapping work. Its bounded runner serves palette routing and Foundation Models voice correction: timeout or caller cancellation returns after a 50 ms cleanup grace, while cancellation-ignoring work retains admission until it exits. Invalid deadlines admit no work; finite deadlines are capped at 60 seconds. LocalCorrectorProbe shares the local correctors' bounded readiness request and typed unavailable/configuration outcomes, while provider adapters retain their route and capability choices.

  • BoundedProcess drains diagnostic tool output without background reader tasks, caps retained output at 1 MiB by default, and bounds waits for process termination and inherited pipe closure. ProcessIdentity accepts only successful, complete, nontruncated output; partial probes remain unavailable.

  • FloatingPanelChrome, HoverTracking and PaletteSelection own the common panel chrome, tracking areas and selectable-row navigation used by application controllers. The merge retains the audit's current-result, search and operation-lifetime checks around those shared helpers.

  • PasteboardBroker.verifySelectionRange is the shared caret-identity policy: proven AX writers may refuse an expand before erase if the selection moved; after HID erase, text and image clipboard paste require the same app and focused element only. Settle-delay Cmd+V aborts log a distinct reason instead of only restoring the trigger.

  • TextInjectionPipeline.insertAfterErase chooses clipboard paste or Unicode keystrokes after the trigger is erased. Unicode typing applies only where AXWriteCapabilityStore.swallowsSyntheticPaste matches: Cursor's exact bundle ID and the VS Code, Insiders and OSS prefixes. Shell-like targets and every other app keep clipboard paste, and empty text inserts nothing.

  • ActivityHistoryStore and ActivityCenterViewController own the Recent Activity window. The controller attaches its observers each time the window is shown (AppKit does not call viewWillAppear for a reopened window), dispatches update notifications on the main thread, shows a dedicated empty and unhealthy-history overlay, and routes actions through AppDelegate so the live HotkeyManager is preserved. Only finite typed ActivitySignals reach disk; localization is deferred to presentation.

  • FillInBuilder is the single generator of fill-in tokens. MacroCatalog routes its single-line, default-value, multiline, popup and section entries through it, so palette tokens and caret stops cannot drift from the builder. fillPart(name:defaultFlag:content:) supports default flags and sanitizes structural delimiters. C-style observer callbacks in HotkeyManager and SelectionMonitor no longer capture instance method references.

These components replace prior copies at their existing call sites. They do not introduce a second expansion coordinator or a second clipboard/recovery policy.

Productivity search and text operations

SnippetSearch owns phrase/field/type/exclusion parsing and matching for both the manager and command palette. SnippetManagerFilter applies one indexed pass to the complete groups and then intersects the resulting IDs with the manager's current group/chip selection. Search bodies are capped at 2,000 characters; queries at 4,096 UTF-8 bytes and 12 terms, with the palette's additional 512-character limit.

SnippetStore.searchCacheIdentity scopes each local revision. Both identity and revision are required for a constant-time cache check; arbitrary group snapshots use a complete Hashable content fingerprint. Each folded index owns a UUID so caller-created indices and locales cannot share query results. Lexical results are cached separately from custom ranking closures. Palette row caches do not share custom rankings, and cached dynamic values are refreshed through the existing command-hit builder.

TextCaseTransform owns the four identifier styles for the palette and both macro syntaxes. Whole-input cursor mapping handles acronym boundaries that cannot be inferred by transforming a prefix alone. PaletteTextOps owns Unicode line splitting, delegates number formatting to SafeMathParser, encodes URL components with the unreserved character set, and enables scalar JSON fragments on read/write. Existing insertion, permission and secret-routing owners remain authoritative.

Search admission uses SnippetSearch.QueryIssue from the same parser as matching. UTF-8 and term overflow reject the complete query before matching; incomplete negative filters reject too. CommandPaletteCatalog.buildRows validates before normalization or cache lookup, and both native search surfaces display the typed reason. Explicit > command queries retain their command grammar.