Skip to content

Compression for S3 Upload / Download #3259

Closed
@dylanbhughes

Description

Current behavior

Inspired by: https://prefect-community.slack.com/archives/CL09KU1K7/p1599160102266700

The S3 tasks in the task library currently don't compress/uncompress data for you. It would be a lot cooler if they did.

Proposed behavior

Add a compression argument to these tasks that could support different methods. Maybe start with gzip?

Example

Sometimes we want to compress info before uploading to S3 to save on upload/download time.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementAn improvement of an existing featuregood first issueThis issue is good for newcomersintegrationsRelated to integrations with other services

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions