Skip to content

DagarDrv/S3Fybe.com

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
 
 
 
 
 
 

Repository files navigation

The script you've been working on is a Python program designed to upload files from a local directory to an AWS S3 bucket while allowing you to specify whether the uploaded files should be made public or private. Additionally, it generates a CSV file containing a list of the uploaded file names and their corresponding S3 URLs for reference.

Here's an extended description of the script:

Importing Libraries: The script begins by importing the necessary libraries:

os: This library is used for handling file and directory operations. boto3: Boto3 is the AWS SDK for Python, which allows interaction with AWS services, and it's used to work with AWS S3. botocore: It's used for configuring the AWS S3 client. logging: The logging module is used to provide informative messages about the progress of the script. File Size Function: The file_size function calculates the size of a file in bytes. It's used to compare the file size of local files with their S3 counterparts to determine whether they should be re-uploaded.

Uploading Files: The uploadFolder function is the core of the script. It does the following:

Validates that the specified local folder exists and is a directory. Walks through the local directory, iterating over files and subdirectories. For each file, it checks if the file with the same name and size already exists in the S3 bucket. If it does, the file is skipped. If the file does not exist or differs in size, it is uploaded to the S3 bucket. You can choose whether to make the uploaded files public or private based on user input. The function also constructs the URL for each uploaded file and stores it in the uploaded_files list. User Interaction: The script prompts the user for the local folder to synchronize and whether to make the uploaded files public. It takes user input for these settings.

Initializing AWS S3 Client: The script initializes the Boto3 S3 client with the provided AWS credentials, S3 endpoint, and region configuration.

Storing Uploaded File URLs: The URLs of the uploaded files, along with their S3 keys, are stored in the uploaded_files list as tuples.

CSV File Creation: The save_links_to_csv function is responsible for creating a CSV file named 'uploaded_files.csv.' It saves the list of uploaded files and their URLs in a structured format. The file contains two columns: "File Name" and "File URL."

Execution and Output: After the synchronization is complete, the script displays the list of uploaded files along with their URLs, and it saves the file links to the CSV file.

In summary, this script provides a user-friendly way to upload and manage files in an AWS S3 bucket. It offers the flexibility to choose the access control level for each uploaded file and generates a CSV file for easy access to the file URLs, which can be particularly useful when sharing the links with others or maintaining a record of uploaded files.

About

Python to upload the data to S3 Fybe.com

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages