Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🎉 v2.0.0 Huge Rewrite #112

Merged
merged 135 commits into from
Apr 23, 2021
Merged

🎉 v2.0.0 Huge Rewrite #112

merged 135 commits into from
Apr 23, 2021

Conversation

sa7mon
Copy link
Owner

@sa7mon sa7mon commented Apr 22, 2021

This is almost a complete re-write of the tool and thus deserves a major version bump.

Changes

  • Check for each combination of open permissions on buckets
  • Check for "dangerous" permissions: Write, WriteACP
  • Simplified the output not have different formats for file and console output. Everything is now just output to stdout in a uniform way to allow easy parsing with grep/awk/etc
  • Supported added for non-AWS S3-compatible APIs. This was done in a generic way to avoid having to include API-specific code in the tool and update it when the APIs inevitably change/break
  • Packaged and uploaded a Pip package for easier distribution
  • Built and pushed a Docker image to Docker Hub: https://hub.docker.com/r/hothamandcheese/s3scanner
  • Increased overall test coverage to ~90%
  • Added support for multi-threaded scanning and dumping
  • Added support for "resume-able" dumping. If an object has already been downloaded, it will be skipped unless the sizes differ
  • Supports Windows/MacOS/Linux with Python 3.6+

Known Issues / Future Work

  • Currently, non-AWS endpoints are only scanned for anonymous permissions. Testing is needed to see if credential scans work and if the permissions match AWS structure.
  • When dumping a bucket, the tool will check to see if each file has already been downloaded. If it has, the file will be skipped unless the size of the local and remote files don't match. In the future, the user should be given a choice to re-download these files.
  • Measure user desire for other output formats (i.e. csv/json/sqlite)

Closes #16
Closes #41
Closes #53
Closes #62
Closes #66
Closes #67
Closes #70
Closes #73
Closes #82
Closes #100

Dan Salmon and others added 30 commits September 30, 2019 01:43
@sa7mon sa7mon merged commit fb39258 into master Apr 23, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment