Folder Comparison and Sync Tool

A high-performance CLI tool to recursively compare and sync two folders using metadata, partial hashing, and full SHA-256 verification.

Features

• Optimized Scanning: Uses a 3-level check (Size → Metadata → Partial Hash) for near-instant comparison of identical folders.
• Deep Verification: Optional full SHA-256 hashing for 100% binary certainty.
• Ignore System Files: Automatically skips desktop.ini, .DS_Store, and Thumbs.db.
• Interactive Sync: Selective copy, overwrite, or delete with arrow-key navigation.
• Modern CLI UI: Premium terminal experience with ANSI colors and progress tracking.

Setup & Run

1. Ensure you have Python 3 installed.
2. Run the script (Quick Mode):

   python compare_dirs.py "path/a" "path/b"

3. Run with Full Hashing (Deep Mode):

   python compare_dirs.py "path/a" "path/b" --deep

Interactive Navigation

• Up/Down Arrows: Move selection.
• Enter: Confirm selection.
• Ctrl+C: Graceful exit.

Sync Logic

• Folder A -> Folder B: Syncs new files and handles selective overwrites.
• Folder B -> Folder A: Syncs new files found only in the second folder.

Performance

• Metadata Trusting: Skips hashing if size and modification time match.
• Partial Hashing: Verifies the first 128KB to detect differences without reading entire large files.
• Buffered I/O: Uses 1MB buffers for maximum disk throughput.

Note on Accuracy: The partial hash method (128KB) is extremely reliable for media files. The only real-world failure case is if two large files were copied from the same source, but one got corrupted after the 128KB mark (e.g., bad sector). In that rare scenario, they would be falsely marked as identical. For 100% certainty, use the --deep flag to perform full SHA-256 hashing.

Bonus: The tool detects subtle differences manual inspection misses—like folders with identical file counts and total sizes, but with files reorganized into different subfolders. It catches these scenarios by comparing content hashes per-file, not just folder-level statistics.

Performance Note: On a dataset with approximately 285,000 files, the tool takes around 250-300 seconds for scanning directories and about 250-400 seconds for the indexing phase (metadata/stat collection) on typical HDD storage. These timings may vary based on disk speed, CPU, and file distribution, but they illustrate the tool's ability to handle large directories efficiently with modern hardware.