Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

see #63 adds multipart upload functions #64

Merged
merged 2 commits into from
Jul 22, 2022
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
see #63 adds multipart upload functions
  • Loading branch information
mark-distrokid committed Jul 19, 2022
commit 4afccd495a87dcf55bb44f9e0d7bdefbaf3aec0d
170 changes: 170 additions & 0 deletions services/s3.cfc
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,172 @@ component {
return apiResponse;
}


/**
* This action initiates a multipart upload and returns an upload ID. This upload ID is used to associate all of the parts in the specific multipart upload. You specify this upload ID in each of your subsequent upload part requests (see UploadPart). You also include this upload ID in the final request to either complete or abort the multipart upload request.
* https://docs.aws.amazon.com/AmazonS3/latest/API/API_CreateMultipartUpload.html
*
*
* @Bucket: The name of the bucket to which to initiate the upload
* @Key: Object key for which the multipart upload is to be initiated.
*/
public any function createMultipartUpload(
required string Bucket,
required string Key
){

var requestSettings = api.resolveRequestSettings( argumentCollection = arguments );

var queryParams = { 'uploads': '' };
for (
var key in arguments
) {
// if ( len( arguments[ key ] ) ) queryParams[ utils.parseKey( key ) ] = arguments[ key ];
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I take it nothing from the arguments needs to go into the query string? Should this be removed then?

}
var apiResponse = apiCall(
requestSettings,
'POST',
'/#arguments.Key#',
queryParams,
{
// "x-amz-acl": "public-read"
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be removed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah! Will remove in my next PR!

}
);
if ( apiResponse.statusCode == 200 ) {
apiResponse[ 'data' ] = utils.parseXmlDocument( apiResponse.rawData );
}
return apiResponse;

}

public any function putPart(
required string Bucket,
required string Key,
required string UploadId,
required numeric PartNumber,
required any body

){
var requestSettings = api.resolveRequestSettings( argumentCollection = arguments );
dump("putPart");
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be removed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep! Will do!

var queryParams = {
'partNumber': arguments.partNumber,
"uploadId": arguments.uploadId
};

for (
var key in arguments
) {
// if ( len( arguments[ key ] ) ) queryParams[ utils.parseKey( key ) ] = arguments[ key ];
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question as above, looks like this argument key loop isn't necessary?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like I had a copy pasta frenzy! removing!

}
var apiResponse = apiCall(
requestSettings,
'PUT',
'/#arguments.Key#',
queryParams,
{
// "x-amz-acl": "public-read"
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be removed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I was debugging hard here :)

},
arguments.body
);

if ( apiResponse.statusCode == 200 ) {
// apiResponse[ 'data' ] = utils.parseXmlDocument( apiResponse.rawData );
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know what the api response looks like here, is it an XML document? If not, can this 200 check be removed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it should be XML for the most part.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually in this case it just returns headers rather than any body content so I had to remove the XML Parsing

}
return apiResponse;

}


public any function completeMultipartUpload(
required string Bucket,
required string Key,
required string UploadId,
required array Parts
){

var requestSettings = api.resolveRequestSettings( argumentCollection = arguments );
var payload = getPartsPayload(Parts);
var queryParams = {
"uploadId": arguments.UploadId
};
for (
var key in arguments
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question again regarding the argument loop 😄

) {
// if(key eq "Parts"){
// continue;
// }

// if ( len( arguments[ key ] ) ) queryParams[ utils.parseKey( key ) ] = arguments[ key ];
}
var apiResponse = apiCall(
requestSettings,
'POST',
'/#arguments.Key#',
queryParams,
{},
toString(payload)

);


if ( apiResponse.statusCode == 200 ) {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question regarding the response status check.

// apiResponse[ 'data' ] = utils.parseXmlDocument( apiResponse.rawData );
}
return apiResponse;

}

private xml function getPartsPayload(required Array Parts){

var ret = '<?xml version="1.0" encoding="UTF-8"?>';
ret &= '<CompleteMultipartUpload xmlns="http://s3.amazonaws.com/doc/2006-03-01/">';

for (var part in arguments.Parts){
ret &= "<Part><ETag>#part.ETag#</ETag><PartNumber>#part.PartNumber#</PartNumber></Part>";
}

ret &= '</CompleteMultipartUpload>'
return XMLParse(ret);
}

// ** modified version of generate presigned. Just for putting uploads

public string function generatePresignedURLForPart(
required string Bucket,
required string Key,
required numeric partNumber,
required string uploadId,
// required string ContentType,
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REmoved. I thought it was needed as part of the signing but apparently not.

numeric Expires = 300,
string VersionId = ''
) {

var requestSettings = api.resolveRequestSettings( argumentCollection = arguments );
var host = getHost( requestSettings );
var path = arguments.Bucket.find( '.' ) ? '/' & arguments.Bucket : '';
path &= '/' & Key;
var queryParams = {
"partNumber": arguments.partNumber,
"uploadId": arguments.uploadId
};
if ( len( arguments.VersionId ) ) queryParams[ 'versionId' ] = arguments.VersionId;


return api.signedUrl(
variables.service,
host,
requestSettings.region,
'PUT',
path,
queryParams,
Expires,
requestSettings.awsCredentials,
false
);
}


/**
* creates a new bucket
* http://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUT.html
Expand Down Expand Up @@ -451,12 +617,15 @@ component {
numeric Expires = 300,
string VersionId = ''
) {


var requestSettings = api.resolveRequestSettings( argumentCollection = arguments );
var host = getHost( requestSettings );
var path = arguments.Bucket.find( '.' ) ? '/' & arguments.Bucket : '';
path &= '/' & ObjectKey;
var queryParams = { };
if ( len( arguments.VersionId ) ) queryParams[ 'versionId' ] = arguments.VersionId;

return api.signedUrl(
variables.service,
host,
Expand Down Expand Up @@ -792,6 +961,7 @@ component {

var useSSL = !structKeyExists( variables.settings, 'useSSL' ) || variables.settings.useSSL;


return api.call(
variables.service,
host,
Expand Down