Update Connection#download method to stream data

[braxtone]

The download method waits for the entire response body before returning the data to the caller or storing it into a file. This change implements streaming the request to accommodate large file/report downloads.

Description

Streams the HTTP Response body to a file, allows for the caller to pass in a block, and maintains the default behaviour of the method to avoid needing changes.

Motivation and Context

I was seeing weird timeout and Errno::EINVAL issues that were thrown when large (4GB) report downloads were requested. This addresses those.

How Has This Been Tested?

Used to download 100 reports of sizes ranging from 5KB to 5GB and successfully completed all of them.

No impact to existing code

Screenshots (if appropriate):

Types of changes

• New feature (non-breaking change which adds functionality)

Checklist:

• I have updated the documentation accordingly (if changes are required).
• I have added tests to cover my changes.
• All new and existing tests passed.

Large file downloads can cause timeout issues and odd Errno::EINVAL errors to be raised. This update provides a way to stream the contents of the response body to a file, yield a block, and maintains the original behavior if desired.

[sgreen-r7]

Looks rad, thank you for this. We'll kick the tires on it internally and look forward to merging it soon.

[gschneider-r7]

Looks like hound's comments don't get hidden when resolved anymore. 😢

[braxtone]

Sounds great Scott. Ha, yeah it looks like my shame will just have to remain documented for the world to see.

[gschneider-r7]

I ended up backing this out because I ran into some other issues while testing it. In summary:

• File.open needs the :: prefix to resolve namespace issues
• filename needs to be file_name to match the input variable
• Saving to a file (e.g. file_name is provided) needs to return the file reference, not the HTTP response

If you want to update this branch or create a new one, be sure to first revert my commit locally (d2c5759) before making any other changes so that the later merge will work correctly.

@sgreen-r7 and I will take another look at this so if you don't get around to it yourself @braxtone don't worry too much about it.

[braxtone]

So sorry for the trouble! I'll sync my fork with your commits and make updates to address those issues.