-
Notifications
You must be signed in to change notification settings - Fork 16
Open
Labels
fix: inaccuracy ⚠️Inaccurate informationInaccurate informationsystem: data-mgmt 🗃️Data management relatedData management relatedsystem: storage 🛢Docs related to storageDocs related to storage
Description
What is inaccurate?
- s3cmd is unmaintained and its documentation recommends using awscli
- s5cmd appears to be abandoned based on time of the last release (14 months ago) and the state of issues and PRs.
Here are our recommendations:
- Prefer Globus (once the performance degradation bug is fixed) in all cases.
- If a CLI tool is required, consider Globus CLI.
- If Globus is not possible...
- ...and the data is from LTS to/from a local machine, a VM, or Cheaha, use awscli and make use of parallelization as needed. rclone may also be used.
- ...and the data is from LTS to AWS, or any other S3-compatible storage, use rclone. awscli is not capable of transferring data between two different S3 endpoints.
As part of making this change, we need to analyze the performance of data transfers using awscli and rclone using comparable parameters. This will guide how we suggest prioritizing which software to choose.
Where is the inaccuracy?
Review all LTS pages, and anywhere else, for mentions of s3cmd and s5cmd.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
fix: inaccuracy ⚠️Inaccurate informationInaccurate informationsystem: data-mgmt 🗃️Data management relatedData management relatedsystem: storage 🛢Docs related to storageDocs related to storage