Skip to content

Updating aws sdk #85

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Updating aws sdk #85

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ad4073a
Updating aws sdk
Oct 5, 2015
d5e732b
Updated to support getting credentials from the IAM role
Apr 20, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion project.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,6 @@
:license {:name "Eclipse Public License"
:url "http://www.eclipse.org/legal/epl-v10.html"}
:dependencies [[org.clojure/clojure "1.2.1"]
[com.amazonaws/aws-java-sdk "1.7.5"]
[com.amazonaws/aws-java-sdk "1.10.22"]
[clj-time "0.6.0"]]
:plugins [[codox "0.8.10"]])
36 changes: 21 additions & 15 deletions src/aws/sdk/s3.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
[clojure.walk :as walk])
(:import com.amazonaws.auth.BasicAWSCredentials
com.amazonaws.auth.BasicSessionCredentials
com.amazonaws.auth.InstanceProfileCredentialsProvider
com.amazonaws.services.s3.AmazonS3Client
com.amazonaws.AmazonServiceException
com.amazonaws.ClientConfiguration
Expand Down Expand Up @@ -72,10 +73,15 @@
(.setProxyDomain client-configuration proxy-domain))
(when-let [proxy-workstation (get-in cred [:proxy :workstation])]
(.setProxyWorkstation client-configuration proxy-workstation))
;;There is another option in that we may want to use the IAM role of the instance
;;to dictate our credentials. For this scenario we use a different cred provider
;;http://docs.aws.amazon.com/AWSSdkDocsJava/latest/DeveloperGuide/java-dg-roles.html
(let [aws-creds
(if (:token cred)
(BasicSessionCredentials. (:access-key cred) (:secret-key cred) (:token cred))
(BasicAWSCredentials. (:access-key cred) (:secret-key cred)))
(if cred
(if (:token cred)
(BasicSessionCredentials. (:access-key cred) (:secret-key cred) (:token cred))
(BasicAWSCredentials. (:access-key cred) (:secret-key cred)))
(InstanceProfileCredentialsProvider.))

client (AmazonS3Client. aws-creds client-configuration)]
(when-let [endpoint (:endpoint cred)]
Expand Down Expand Up @@ -217,26 +223,26 @@
(.putObject (s3-client cred) req)))

(defn- initiate-multipart-upload
[cred bucket key]
(.getUploadId (.initiateMultipartUpload
(s3-client cred)
[cred bucket key]
(.getUploadId (.initiateMultipartUpload
(s3-client cred)
(InitiateMultipartUploadRequest. bucket key))))

(defn- abort-multipart-upload
[{cred :cred bucket :bucket key :key upload-id :upload-id}]
(.abortMultipartUpload
(s3-client cred)
[{cred :cred bucket :bucket key :key upload-id :upload-id}]
(.abortMultipartUpload
(s3-client cred)
(AbortMultipartUploadRequest. bucket key upload-id)))

(defn- complete-multipart-upload
[{cred :cred bucket :bucket key :key upload-id :upload-id e-tags :e-tags}]
[{cred :cred bucket :bucket key :key upload-id :upload-id e-tags :e-tags}]
(.completeMultipartUpload
(s3-client cred)
(CompleteMultipartUploadRequest. bucket key upload-id e-tags)))

(defn- upload-part
[{cred :cred bucket :bucket key :key upload-id :upload-id
part-size :part-size offset :offset ^java.io.File file :file}]
part-size :part-size offset :offset ^java.io.File file :file}]
(.getPartETag
(.uploadPart
(s3-client cred)
Expand All @@ -251,8 +257,8 @@

(defn put-multipart-object
"Do a multipart upload of a file into a S3 bucket at the specified key.
The value must be a java.io.File object. The entire file is uploaded
or not at all. If an exception happens at any time the upload is aborted
The value must be a java.io.File object. The entire file is uploaded
or not at all. If an exception happens at any time the upload is aborted
and the exception is rethrown. The size of the parts and the number of
threads uploading the parts can be configured in the last argument as a
map with the following keys:
Expand All @@ -271,8 +277,8 @@
(try
(complete-multipart-upload
(assoc upload :e-tags (map #(.get ^java.util.concurrent.Future %) (.invokeAll pool tasks))))
(catch Exception ex
(abort-multipart-upload upload)
(catch Exception ex
(abort-multipart-upload upload)
(.shutdown pool)
(throw ex))
(finally (.shutdown pool)))))
Expand Down