Skip to content

feat: Add Google Drive storage backend - #8

Merged
samscarrow merged 4 commits into
mainfrom
feat/google-drive-storage
Aug 18, 2025
Merged

feat: Add Google Drive storage backend#8
samscarrow merged 4 commits into
mainfrom
feat/google-drive-storage

Conversation

@samscarrow

Copy link
Copy Markdown
Collaborator

Summary

  • Implements complete Google Drive storage backend for PhotoSight
  • Adds chunked upload support for large RAW files (20-50MB)
  • Provides comprehensive Drive operations (upload, download, search, share)

Key Features

Storage Manager (gdrive_manager.py)

  • Service account authentication
  • Folder management with date-based organization
  • Single and batch upload operations
  • Search with filters (date range, query)
  • Download and sharing capabilities
  • Storage quota monitoring

Chunked Upload (chunked_upload.py)

  • 5MB chunk size optimized for RAW files
  • Automatic retry with exponential backoff
  • MD5 hash verification for data integrity
  • Parallel batch uploads (configurable concurrency)
  • Progress tracking for large files

Configuration

  • Environment-based configuration (no hardcoded secrets)
  • Service account JSON file path from env variable
  • Customizable root folder name
  • Configurable retry and backoff parameters

Technical Details

  • Uses Google Drive API v3
  • Implements resumable uploads for reliability
  • Handles rate limiting gracefully
  • Supports all major RAW formats (ARW, CR2, NEF, DNG)
  • Thread-safe batch operations

Testing

  • Verify service account setup
  • Test single file upload/download
  • Test batch operations
  • Confirm retry logic on network failures

Dependencies Added

  • google-api-python-client>=2.100.0
  • google-auth>=2.23.0
  • google-auth-httplib2>=0.1.1
  • google-auth-oauthlib>=1.1.0

🤖 Generated with Claude Code

- Add GoogleDriveManager for comprehensive Drive operations
- Implement chunked upload with retry logic for large RAW files
- Add batch upload support with parallel processing
- Include date-based folder organization
- Add search, download, and sharing capabilities
- Configure 5MB chunks with exponential backoff for reliability
- Add Google Drive dependencies to requirements.txt

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 18, 2025 01:12
@github-actions

Copy link
Copy Markdown

🤖 Claude Code Review Error

Sorry, I encountered an error while reviewing this PR:

Please check the workflow logs for more details.

@github-actions

Copy link
Copy Markdown

Claude Code Review Failed: Please check the workflow logs for details.

@socket-security

socket-security Bot commented Aug 18, 2025

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedpypi/​google-auth@​2.40.394100100100100
Addedpypi/​python-dotenv@​1.1.199100100100100
Addedpypi/​google-api-python-client@​2.179.099100100100100
Addedpypi/​google-auth-oauthlib@​1.2.299100100100100
Addedpypi/​google-auth-httplib2@​0.2.0100100100100100

View full report

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Implements a complete Google Drive storage backend for PhotoSight to handle photo storage and management. The implementation focuses on handling large RAW files (20-50MB) with reliable upload mechanisms and comprehensive Drive operations.

  • Adds chunked upload support with retry logic and exponential backoff for reliable large file transfers
  • Implements date-based folder organization and comprehensive Drive operations (upload, download, search, share)
  • Provides batch upload capabilities with configurable parallelism for processing multiple photos

Reviewed Changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 4 comments.

File Description
photosight/storage/gdrive_manager.py Main Google Drive manager with authentication, folder management, and all Drive operations
photosight/storage/chunked_upload.py Chunked upload implementation with retry logic and batch processing for large RAW files
photosight/storage/init.py Package initialization exposing key classes

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
You can also share your feedback on Copilot code review for a chance to win a $100 gift card. Take the survey.

Comment thread photosight/storage/gdrive_manager.py Outdated
Comment thread photosight/storage/chunked_upload.py Outdated
Comment thread photosight/storage/gdrive_manager.py
Comment thread photosight/storage/gdrive_manager.py Outdated
samscarrow and others added 3 commits August 17, 2025 21:13
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown

🤖 Claude Code Review Error

Sorry, I encountered an error while reviewing this PR:

Please check the workflow logs for more details.

@github-actions

Copy link
Copy Markdown

Claude Code Review Failed: Please check the workflow logs for details.

@github-actions

Copy link
Copy Markdown

🤖 Claude Code Review Error

Sorry, I encountered an error while reviewing this PR:

Please check the workflow logs for more details.

@github-actions

Copy link
Copy Markdown

Claude Code Review Failed: Please check the workflow logs for details.

@github-actions

Copy link
Copy Markdown

🤖 Claude Code Review Error

Sorry, I encountered an error while reviewing this PR:

Please check the workflow logs for more details.

@github-actions

Copy link
Copy Markdown

Claude Code Review Failed: Please check the workflow logs for details.

@samscarrow
samscarrow merged commit 60df446 into main Aug 18, 2025
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants