Improve recording stability and A/V sync #1478

richiemcilroy · 2025-12-31T18:19:19Z

Adds drift correction for audio and video timestamps to prevent sync issues during long recordings, resolution downscaling for hardware encoder limits, and retry logic for Windows file operations. Also filters incomplete recordings by cutoff date in recovery and removes verbose camera trace logs.

Greptile Summary

Implements comprehensive drift correction for audio/video timestamps to prevent sync issues during long recordings by tracking wall clock vs stream time divergence
Adds resolution downscaling for hardware encoder compatibility to prevent recording failures when capture resolution exceeds H.264 limits (4096px max dimension)
Introduces Windows-specific retry logic for file operations and filters old incomplete recordings from recovery to improve stability and performance

Important Files Changed

Filename	Overview
`crates/recording/src/output_pipeline/core.rs`	Added AudioDriftTracker and VideoDriftTracker with extensive testing; implements baseline offset capture and drift ratio corrections for A/V sync
`crates/recording/src/studio_recording.rs`	Refactored audio muxer from segmented to fragmented approach; added GPU-compatible resolution calculation and changed audio output to single files
`crates/recording/src/sources/audio_mixer.rs`	Added drift correction tracking per audio source with wall clock vs audio duration comparison to maintain sync during extended recordings

Confidence score: 4/5

This PR introduces complex timing and synchronization logic that requires careful testing to ensure it works correctly across different hardware configurations and recording durations
Score reflects the comprehensive nature of the changes affecting core recording infrastructure, though the implementation appears well-structured with good test coverage and safety bounds
Pay close attention to the drift correction algorithms in core.rs and audio_mixer.rs as these directly impact A/V synchronization quality

Sequence Diagram

sequenceDiagram
    participant User
    participant RecordingModule as "recording.rs"
    participant SegmentedVideoEncoder as "SegmentedVideoEncoder"
    participant H264Encoder
    participant AudioMixer
    participant DriftTracker as "Audio/VideoDriftTracker"
    participant RecoveryManager
    
    User->>RecordingModule: "start_recording(inputs)"
    
    RecordingModule->>RecordingModule: "validate state & create project dir"
    RecordingModule->>RecordingModule: "write initial RecordingMeta"
    RecordingModule->>SegmentedVideoEncoder: "init(path, video_config, config)"
    SegmentedVideoEncoder->>H264Encoder: "build(&mut output)"
    H264Encoder-->>SegmentedVideoEncoder: "encoder instance"
    SegmentedVideoEncoder-->>RecordingModule: "encoder ready"
    
    RecordingModule->>AudioMixer: "setup(audio_sources)"
    AudioMixer-->>RecordingModule: "mixer ready"
    
    RecordingModule->>RecordingModule: "emit RecordingEvent::Started"
    
    loop "Recording Loop"
        RecordingModule->>SegmentedVideoEncoder: "queue_frame(frame, timestamp)"
        SegmentedVideoEncoder->>DriftTracker: "calculate_timestamp(raw_duration, wall_clock)"
        DriftTracker-->>SegmentedVideoEncoder: "corrected_timestamp"
        SegmentedVideoEncoder->>H264Encoder: "queue_frame(frame, corrected_timestamp)"
        H264Encoder-->>SegmentedVideoEncoder: "frame encoded"
        
        RecordingModule->>AudioMixer: "send audio frames"
        AudioMixer->>DriftTracker: "calculate_timestamp(samples, sample_rate, wall_clock)"
        DriftTracker-->>AudioMixer: "corrected_timestamp"
        AudioMixer-->>RecordingModule: "mixed audio frame"
        
        alt "Segment Complete"
            SegmentedVideoEncoder->>SegmentedVideoEncoder: "on_segment_completed()"
            SegmentedVideoEncoder->>SegmentedVideoEncoder: "write_manifest()"
        end
    end
    
    User->>RecordingModule: "stop_recording()"
    RecordingModule->>SegmentedVideoEncoder: "finish(timestamp)"
    SegmentedVideoEncoder->>H264Encoder: "flush()"
    SegmentedVideoEncoder->>SegmentedVideoEncoder: "finalize_manifest()"
    SegmentedVideoEncoder-->>RecordingModule: "recording finished"
    
    alt "Fragmented Recording (if interrupted)"
        RecordingModule->>RecoveryManager: "find_incomplete(recordings_dir)"
        RecoveryManager-->>RecordingModule: "incomplete recordings list"
        User->>RecordingModule: "recover_recording(project_path)"
        RecordingModule->>RecoveryManager: "recover(recording)"
        RecoveryManager-->>RecordingModule: "recovered recording"
    end
    
    RecordingModule->>RecordingModule: "handle_recording_finish()"
    RecordingModule->>RecordingModule: "emit RecordingEvent::Stopped"

Context used:

Context from dashboard - CLAUDE.md (source)
Context from dashboard - AGENTS.md (source)

greptile-apps

_{10 files reviewed, 5 comments}

_{Edit Code Review Agent Settings | Greptile}

greptile-apps · 2025-12-31T18:26:04Z

crates/recording/src/output_pipeline/ffmpeg.rs

+        let adjusted_timestamp = if let Some(pause) = &self.pause {
+            match pause.adjust(timestamp)? {
+                Some(ts) => ts,
+                None => return Ok(()),
+            }
+        } else {
+            timestamp
+        };


style: Frame skipping logic returns Ok(()) when pause.adjust() returns None - ensure this doesn't cause audio gaps or sync drift when resuming. Should there be logging when audio frames are skipped during pause to help debug sync issues?

_{Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!}

Prompt To Fix With AI

This is a comment left during a code review. Path: crates/recording/src/output_pipeline/ffmpeg.rs Line: 224:231 Comment: **style:** Frame skipping logic returns Ok(()) when pause.adjust() returns None - ensure this doesn't cause audio gaps or sync drift when resuming. Should there be logging when audio frames are skipped during pause to help debug sync issues? Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time! How can I resolve this? If you propose a fix, please make it concise.

greptile-apps · 2025-12-31T18:26:04Z

apps/desktop/src-tauri/src/recovery.rs


 use crate::create_screenshot;

+const RECOVERY_CUTOFF_DATE: (i32, u32, u32) = (2025, 12, 31);


logic: The cutoff date is in the future (2025-12-31 vs current date 2025-12-31). This means the filter will currently exclude all recordings since no recordings can be created after today's date. Should the cutoff date be in the past to filter out old recordings, or is this intentionally filtering all current recordings?

Prompt To Fix With AI

This is a comment left during a code review. Path: apps/desktop/src-tauri/src/recovery.rs Line: 12:12 Comment: **logic:** The cutoff date is in the future (2025-12-31 vs current date 2025-12-31). This means the filter will currently exclude all recordings since no recordings can be created after today's date. Should the cutoff date be in the past to filter out old recordings, or is this intentionally filtering all current recordings? How can I resolve this? If you propose a fix, please make it concise.

greptile-apps · 2025-12-31T18:26:05Z

apps/desktop/src-tauri/src/recovery.rs

+fn parse_recording_date(pretty_name: &str) -> Option<NaiveDate> {
+    let date_part = pretty_name.strip_prefix("Cap ")?;
+    let date_str = date_part.split(" at ").next()?;
+    NaiveDate::parse_from_str(date_str, "%Y-%m-%d").ok()
+}


style: Parsing relies on exact string format "Cap YYYY-MM-DD at ..." which creates fragile coupling to the recording naming convention

_{Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!}

Prompt To Fix With AI

This is a comment left during a code review. Path: apps/desktop/src-tauri/src/recovery.rs Line: 14:18 Comment: **style:** Parsing relies on exact string format "Cap YYYY-MM-DD at ..." which creates fragile coupling to the recording naming convention Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time! How can I resolve this? If you propose a fix, please make it concise.

greptile-apps · 2025-12-31T18:26:06Z

apps/desktop/src-tauri/src/recovery.rs

+        RECOVERY_CUTOFF_DATE.1,
+        RECOVERY_CUTOFF_DATE.2,
+    )
+    .expect("Invalid cutoff date");


style: The .expect() will panic if the hardcoded date constants are invalid, but this could be validated at compile time or replaced with a safer approach

_{Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!}

Prompt To Fix With AI

This is a comment left during a code review. Path: apps/desktop/src-tauri/src/recovery.rs Line: 29:29 Comment: **style:** The `.expect()` will panic if the hardcoded date constants are invalid, but this could be validated at compile time or replaced with a safer approach Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time! How can I resolve this? If you propose a fix, please make it concise.

greptile-apps · 2025-12-31T18:26:06Z

crates/enc-ffmpeg/src/mux/segmented_stream.rs

+        const MAX_RETRIES: u32 = 10;
+        const RETRY_DELAY_MS: u64 = 50;


style: Consider making MAX_RETRIES and RETRY_DELAY_MS configurable parameters rather than constants. Are these retry values based on empirical testing, or would making them configurable be beneficial for different environments?

_{Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!}

Prompt To Fix With AI

This is a comment left during a code review. Path: crates/enc-ffmpeg/src/mux/segmented_stream.rs Line: 607:608 Comment: **style:** Consider making MAX_RETRIES and RETRY_DELAY_MS configurable parameters rather than constants. Are these retry values based on empirical testing, or would making them configurable be beneficial for different environments? Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time! How can I resolve this? If you propose a fix, please make it concise.

richiemcilroy added 5 commits December 31, 2025 13:01

Add resolution downscaling for hardware encoder limits

8abd14f

Add retry logic for file rename on Windows

938c3ad

Filter incomplete recordings by cutoff date

895ff96

Remove verbose trace logs from camera feed

ad16fac

Add drift correction for audio and video timestamps

2a72652

greptile-apps bot reviewed Dec 31, 2025

View reviewed changes

richiemcilroy added 10 commits December 31, 2025 18:58

fmt

993d7d9

clippy

4cc05c7

fmt

1ca0ae7

Optimize device queries and refactor permissions handling

a6b591a

Make recovery scan non-blocking and fix window show logic

327311b

Add camera window visibility indicator to CameraSelect

b20af53

fmt

d38896f

Refactor RecoveryToast to use local signal for recordings

060290c

Add device snapshot API and event emitter

d0c7f87

Refactor device and permission queries to use unified snapshot

1c7c565

richiemcilroy merged commit 6d6c67d into main Dec 31, 2025
16 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Improve recording stability and A/V sync #1478

Improve recording stability and A/V sync #1478

richiemcilroy commented Dec 31, 2025 •

edited by greptile-apps bot

Loading

Uh oh!

greptile-apps bot left a comment

Uh oh!

greptile-apps bot Dec 31, 2025

Uh oh!

greptile-apps bot Dec 31, 2025

Uh oh!

greptile-apps bot Dec 31, 2025

Uh oh!

greptile-apps bot Dec 31, 2025

Uh oh!

greptile-apps bot Dec 31, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants


		use crate::create_screenshot;

		const RECOVERY_CUTOFF_DATE: (i32, u32, u32) = (2025, 12, 31);

Improve recording stability and A/V sync #1478

Improve recording stability and A/V sync #1478

Conversation

richiemcilroy commented Dec 31, 2025 • edited by greptile-apps bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Important Files Changed

Confidence score: 4/5

Sequence Diagram

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Dec 31, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Dec 31, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Dec 31, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Dec 31, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Dec 31, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

richiemcilroy commented Dec 31, 2025 •

edited by greptile-apps bot

Loading