Merge pull request #459 from Azure/dev

Dev

Author: rickle-msft

BreakingChanges.txt

@@ -1,8 +1,20 @@
-2018.08.11 Version 10.1.0
+XXXX.XX.XX Version XX.X.X
+* Upgraded to version 2.1.0 of the autorest-clientruntime which includes several important fixes to mitigate a commonly-seen "Connection reset by peer" error and other similar bugs.
+* HTTPGetterInfo was made an internal type as it is an internal implementation detail.
+* Added SourceModifiedAccessConditions parameter to StageBlockFromURL
+* Removed DEFAULT and NONE static variables. Empty constructors should be used instead. DEFAULT static values were error prone and unsafe to use because although the field was final, the objects were mutable, so it was possible the value could be changed accidentally and alter the behavior of the program.
+* Changed *ListingDetails to *ListDetails. These name changes are to mitigate conflicts with v8, allowing for side-by-side loading of different versions, which may help with upgrading.
+* Removed the extra quotes around etags in some responses so they are consistently now consistently formatted.
+* Moved the Generated*** types into the blob package to avoid conflicts with generated types from other services (i.e. queues and files)
+* Changed the logger name to be the name of class that uses it, which is a more conventional practice
+* Made the max results field on listing responses Integer instead of int as it is an optional field only returned when specified in the request.
+
+2018.08.11 Version 10.1.0
 * Interfaces for helper types updated to be more consistent throughout the library. All types, with the exception of the options for pipeline factories, use a fluent pattern.
 * Removed RetryReader type as it's functionality was moved to be built into the DownloadResponse. RetryReaderOptions are now named ReliableDownloadOptions.
 * Restructured the access conditions to be more logically adhere to their respective functions.
 * Added support for context parameter on each api to allow communication with the pipeline from the application level
+* Added an option to configure that maximum size data that will be uploaded in a single shot via the TransferManager.
 
 2018.08.22 Version 10.0.4-rc
 * Changed BlobURL.startCopy sourceAccessConditions parameter to be HTTPAccessConditions as lease is not actually supported.

ChangeLog.txt

@@ -1,3 +1,23 @@
+2019.03.22 Version 11.0.0
+* Upgraded to version 2.1.0 of the autorest-clientruntime which includes several important fixes to mitigate a commonly-seen "Connection reset by peer" error and other similar bugs.
+* Support for 2018-11-09 REST version. Please see our REST API documentation and blogs for information about the related added features.
+* Added appendBlockFromURL method. A block may be created with another blob as its source.
+* Added uploadPagesFromURL method. Pages may be written to with another blob as their source.
+* Fixed a bug that would set an invalid range header when downloading an empty blob.
+* Modified the LoggingFactory to redact SAS signatures on outgoing requests.
+* HTTPGetterInfo was made an internal type as it is an internal implementation detail.
+* Removed DEFAULT and NONE static variables. Empty constructors should be used instead. DEFAULT static values were error prone and unsafe to use because although the field was final, the objects were mutable, so it was possible the value could be changed accidentally and alter the behavior of the program.
+* Optimized the TransferManager download to file method to skip the initial HEAD request.
+* Added an option to configure that maximum size data that will be uploaded in a single shot via the TransferManager.
+* Added request Http Method, URL, and headers to logging messages.
+* Changed *ListingDetails to *ListDetails. These name changes are to mitigate conflicts with v8, allowing for side-by-side loading of different versions, which may help with upgrading.
+* Removed the extra quotes around etags in some responses so they are consistently now consistently formatted.
+* Moved the Generated*** types into the blob package to avoid conflicts with generated types from other services (i.e. queues and files)
+* Changed the logger name to be the name of class that uses it, which is a more conventional practice
+* Support added for SAS tokens to scope to blob snapshot.
+* Added getUserDelegationKey to ServiceURL, the result of which can be used to generate a user-delegation SAS.
+* Made the max results field on listing responses Integer instead of int as it is an optional field only returned when specified in the request.
+
 2019.02.15 Version 10.5.0
 * Added uploadFromNonReplayableFlowable to support uploading arbitrary data sources (like network streams) to a block blob.
 

README.md

@@ -37,7 +37,7 @@ To get the binaries of this library as distributed by Microsoft, ready for use w
 <dependency>
 	<groupId>com.microsoft.azure</groupId>
 	<artifactId>azure-storage-blob</artifactId>
-	<version>10.5.0</version>
+	<version>11.0.0</version>
 </dependency>
 ```
 

pom.xml

@@ -14,7 +14,7 @@
 
     <groupId>com.microsoft.azure</groupId>
     <artifactId>azure-storage-blob</artifactId>
-    <version>10.5.0</version>
+    <version>11.0.0</version>
 
     <name>Azure Storage Blob</name>
     <description>The Azure Storage Java Blob library.</description>
@@ -65,13 +65,19 @@
         <dependency>
             <groupId>com.microsoft.rest.v2</groupId>
             <artifactId>client-runtime</artifactId>
-            <version>2.0.2</version>
+            <version>2.1.0</version>
         </dependency>
         <dependency>
             <groupId>org.slf4j</groupId>
             <artifactId>slf4j-api</artifactId>
             <version>1.7.25</version>
         </dependency>
+        <dependency>
+            <groupId>com.microsoft.azure</groupId>
+            <artifactId>adal4j</artifactId>
+            <version>1.6.3</version>
+            <scope>test</scope>
+        </dependency>
         <dependency>
             <groupId>junit</groupId>
             <artifactId>junit</artifactId>

src/main/java/com/microsoft/azure/storage/blob/AccountSASSignatureValues.java

@@ -217,7 +217,7 @@ public SASQueryParameters generateSASQueryParameters(SharedKeyCredentials shared
 
         return new SASQueryParameters(this.version, this.services, resourceTypes,
                 this.protocol, this.startTime, this.expiryTime, this.ipRange, null,
-                null, this.permissions, signature, null, null, null, null, null);
+                null, this.permissions, signature, null, null, null, null, null, null);
     }
 
     private String stringToSign(final SharedKeyCredentials sharedKeyCredentials) {
@@ -228,7 +228,7 @@ private String stringToSign(final SharedKeyCredentials sharedKeyCredentials) {
                 resourceTypes,
                 this.startTime == null ? "" : Utility.ISO8601UTCDateFormatter.format(this.startTime),
                 Utility.ISO8601UTCDateFormatter.format(this.expiryTime),
-                this.ipRange == null ? IPRange.DEFAULT.toString() : this.ipRange.toString(),
+                this.ipRange == null ? (new IPRange()).toString() : this.ipRange.toString(),
                 this.protocol == null ? "" : this.protocol.toString(),
                 this.version,
                 Constants.EMPTY_STRING // Account SAS requires an additional newline character

src/main/java/com/microsoft/azure/storage/blob/AppendBlobAccessConditions.java

@@ -28,12 +28,6 @@
  */
 public final class AppendBlobAccessConditions {
 
-    /**
-     * An object representing no access conditions.
-     */
-    public static final AppendBlobAccessConditions NONE =
-            new AppendBlobAccessConditions();
-
     private AppendPositionAccessConditions appendPositionAccessConditions;
 
     private ModifiedAccessConditions modifiedAccessConditions;

src/main/java/com/microsoft/azure/storage/blob/AppendBlobURL.java

@@ -14,9 +14,7 @@
  */
 package com.microsoft.azure.storage.blob;
 
-import com.microsoft.azure.storage.blob.models.AppendBlobAppendBlockResponse;
-import com.microsoft.azure.storage.blob.models.AppendBlobCreateResponse;
-import com.microsoft.azure.storage.blob.models.BlobHTTPHeaders;
+import com.microsoft.azure.storage.blob.models.*;
 import com.microsoft.rest.v2.Context;
 import com.microsoft.rest.v2.http.HttpPipeline;
 import io.reactivex.Flowable;
@@ -27,7 +25,7 @@
 import java.net.UnknownHostException;
 import java.nio.ByteBuffer;
 
-import static com.microsoft.azure.storage.blob.Utility.addErrorWrappingToSingle;
+import static com.microsoft.azure.storage.blob.Utility.postProcessResponse;
 
 
 /**
@@ -132,11 +130,11 @@ public Single<AppendBlobCreateResponse> create() {
      */
     public Single<AppendBlobCreateResponse> create(BlobHTTPHeaders headers, Metadata metadata,
             BlobAccessConditions accessConditions, Context context) {
-        metadata = metadata == null ? Metadata.NONE : metadata;
-        accessConditions = accessConditions == null ? BlobAccessConditions.NONE : accessConditions;
+        metadata = metadata == null ? new Metadata() : metadata;
+        accessConditions = accessConditions == null ? new BlobAccessConditions() : accessConditions;
         context = context == null ? Context.NONE : context;
 
-        return addErrorWrappingToSingle(this.storageClient.generatedAppendBlobs().createWithRestResponseAsync(context,
+        return postProcessResponse(this.storageClient.generatedAppendBlobs().createWithRestResponseAsync(context,
                 0, null, metadata, null, headers, accessConditions.leaseAccessConditions(),
                 accessConditions.modifiedAccessConditions()));
     }
@@ -195,15 +193,88 @@ public Single<AppendBlobAppendBlockResponse> appendBlock(Flowable<ByteBuffer> da
      */
     public Single<AppendBlobAppendBlockResponse> appendBlock(Flowable<ByteBuffer> data, long length,
             AppendBlobAccessConditions appendBlobAccessConditions, Context context) {
-        appendBlobAccessConditions = appendBlobAccessConditions == null ? AppendBlobAccessConditions.NONE :
+        appendBlobAccessConditions = appendBlobAccessConditions == null ? new AppendBlobAccessConditions() :
                 appendBlobAccessConditions;
         appendBlobAccessConditions = appendBlobAccessConditions == null
-                ? AppendBlobAccessConditions.NONE : appendBlobAccessConditions;
+                ? new AppendBlobAccessConditions() : appendBlobAccessConditions;
         context = context == null ? Context.NONE : context;
 
-        return addErrorWrappingToSingle(this.storageClient.generatedAppendBlobs().appendBlockWithRestResponseAsync(
+        return postProcessResponse(this.storageClient.generatedAppendBlobs().appendBlockWithRestResponseAsync(
                 context, data, length, null, null, null, appendBlobAccessConditions.leaseAccessConditions(),
                 appendBlobAccessConditions.appendPositionAccessConditions(),
                 appendBlobAccessConditions.modifiedAccessConditions()));
     }
+
+    /**
+     * Commits a new block of data from another blob to the end of this append blob. For more information, see the
+     * <a href="https://docs.microsoft.com/rest/api/storageservices/append-block">Azure Docs</a>.
+     * <p>
+     *
+     * @param sourceURL
+     *          The url to the blob that will be the source of the copy.  A source blob in the same storage account can
+     *          be authenticated via Shared Key. However, if the source is a blob in another account, the source blob
+     *          must either be public or must be authenticated via a shared access signature. If the source blob is
+     *          public, no authentication is required to perform the operation.
+     * @param sourceRange
+     *          The source {@link BlobRange} to copy.
+     *
+     * @return Emits the successful response.
+     *
+     * @apiNote ## Sample Code \n
+     * [!code-java[Sample_Code](../azure-storage-java/src/test/java/com/microsoft/azure/storage/Samples.java?name=append_from_url "Sample code for AppendBlobURL.appendBlockFromUrl")]
+     * For more samples, please see the [Samples file](%https://github.com/Azure/azure-storage-java/blob/master/src/test/java/com/microsoft/azure/storage/Samples.java)
+     */
+    public Single<AppendBlobAppendBlockFromUrlResponse> appendBlockFromUrl(URL sourceURL, BlobRange sourceRange) {
+        return this.appendBlockFromUrl(sourceURL, sourceRange, null, null,
+                 null, null);
+    }
+
+    /**
+     * Commits a new block of data from another blob to the end of this append blob. For more information, see the
+     * <a href="https://docs.microsoft.com/rest/api/storageservices/append-block">Azure Docs</a>.
+     * <p>
+     *
+     * @param sourceURL
+     *          The url to the blob that will be the source of the copy.  A source blob in the same storage account can
+     *          be authenticated via Shared Key. However, if the source is a blob in another account, the source blob
+     *          must either be public or must be authenticated via a shared access signature. If the source blob is
+     *          public, no authentication is required to perform the operation.
+     * @param sourceRange
+     *          {@link BlobRange}
+     * @param sourceContentMD5
+     *          An MD5 hash of the block content from the source blob. If specified, the service will calculate the MD5
+     *          of the received data and fail the request if it does not match the provided MD5.
+     * @param destAccessConditions
+     *          {@link AppendBlobAccessConditions}
+     * @param sourceAccessConditions
+     *          {@link SourceModifiedAccessConditions}
+     * @param context
+     *          {@code Context} offers a means of passing arbitrary data (key/value pairs) to an
+     *          {@link com.microsoft.rest.v2.http.HttpPipeline}'s policy objects. Most applications do not need to pass
+     *          arbitrary data to the pipeline and can pass {@code Context.NONE} or {@code null}. Each context object is
+     *          immutable. The {@code withContext} with data method creates a new {@code Context} object that refers to
+     *          its parent, forming a linked list.
+     *
+     * @return Emits the successful response.
+     *
+     * @apiNote ## Sample Code \n
+     * [!code-java[Sample_Code](../azure-storage-java/src/test/java/com/microsoft/azure/storage/Samples.java?name=append_from_url "Sample code for AppendBlobURL.appendBlockFromUrl")]
+     * For more samples, please see the [Samples file](%https://github.com/Azure/azure-storage-java/blob/master/src/test/java/com/microsoft/azure/storage/Samples.java)
+     */
+    public Single<AppendBlobAppendBlockFromUrlResponse> appendBlockFromUrl(URL sourceURL, BlobRange sourceRange,
+            byte[] sourceContentMD5, AppendBlobAccessConditions destAccessConditions,
+            SourceModifiedAccessConditions sourceAccessConditions, Context context) {
+
+        sourceRange = sourceRange == null ? new BlobRange() : sourceRange;
+        destAccessConditions = destAccessConditions == null
+                ? new AppendBlobAccessConditions() : destAccessConditions;
+        context = context == null ? Context.NONE : context;
+
+        return postProcessResponse(
+                this.storageClient.generatedAppendBlobs().appendBlockFromUrlWithRestResponseAsync(context,
+                        sourceURL, 0, sourceRange.toString(), sourceContentMD5, null, null,
+                        destAccessConditions.leaseAccessConditions(),
+                        destAccessConditions.appendPositionAccessConditions(),
+                        destAccessConditions.modifiedAccessConditions(), sourceAccessConditions));
+    }
 }

src/main/java/com/microsoft/azure/storage/blob/BlobAccessConditions.java

@@ -25,9 +25,6 @@
  */
 public final class BlobAccessConditions {
 
-    public static final BlobAccessConditions NONE =
-            new BlobAccessConditions();
-
     private ModifiedAccessConditions modifiedAccessConditions;
 
     private LeaseAccessConditions leaseAccessConditions;

src/main/java/com/microsoft/azure/storage/blob/BlobListingDetails.java renamed to src/main/java/com/microsoft/azure/storage/blob/BlobListDetails.java

@@ -24,12 +24,7 @@
  * changing the details for a different listing operation requires construction of a new object. Null may be passed if
  * none of the options are desirable.
  */
-public final class BlobListingDetails {
-
-    /**
-     * An object representing no listing details.
-     */
-    public static final BlobListingDetails NONE = new BlobListingDetails();
+public final class BlobListDetails {
 
     private boolean copy;
 
@@ -41,7 +36,7 @@ public final class BlobListingDetails {
 
     private boolean deletedBlobs;
 
-    public BlobListingDetails() {
+    public BlobListDetails() {
     }
 
     /**
@@ -56,7 +51,7 @@ public boolean copy() {
      * Whether blob metadata related to any current or previous Copy Blob operation should be included in the
      * response.
      */
-    public BlobListingDetails withCopy(boolean copy) {
+    public BlobListDetails withCopy(boolean copy) {
         this.copy = copy;
         return this;
     }
@@ -71,7 +66,7 @@ public boolean metadata() {
     /**
      * Whether blob metadata should be returned.
      */
-    public BlobListingDetails withMetadata(boolean metadata) {
+    public BlobListDetails withMetadata(boolean metadata) {
         this.metadata = metadata;
         return this;
     }
@@ -86,7 +81,7 @@ public boolean snapshots() {
     /**
      * Whether snapshots should be returned. Snapshots are listed from oldest to newest.
      */
-    public BlobListingDetails withSnapshots(boolean snapshots) {
+    public BlobListDetails withSnapshots(boolean snapshots) {
         this.snapshots = snapshots;
         return this;
     }
@@ -103,7 +98,7 @@ public boolean uncommittedBlobs() {
      * Whether blobs for which blocks have been uploaded, but which have not been committed using Put Block List,
      * should be included in the response.
      */
-    public BlobListingDetails withUncommittedBlobs(boolean uncommittedBlobs) {
+    public BlobListDetails withUncommittedBlobs(boolean uncommittedBlobs) {
         this.uncommittedBlobs = uncommittedBlobs;
         return this;
     }
@@ -118,7 +113,7 @@ public boolean deletedBlobs() {
     /**
      * Whether blobs which have been soft deleted should be returned.
      */
-    public BlobListingDetails withDeletedBlobs(boolean deletedBlobs) {
+    public BlobListDetails withDeletedBlobs(boolean deletedBlobs) {
         this.deletedBlobs = deletedBlobs;
         return this;
     }

src/main/java/com/microsoft/azure/storage/blob/BlobRange.java

@@ -23,11 +23,6 @@
  */
 public final class BlobRange {
 
-    /**
-     * An object which reflects the service's default range, which is the whole blob.
-     */
-    public static final BlobRange DEFAULT = new BlobRange();
-
     private long offset;
 
     private Long count;
@@ -86,4 +81,17 @@ public String toString() {
         return String.format(
                 Locale.ROOT, Constants.HeaderConstants.BEGIN_RANGE_HEADER_FORMAT, this.offset);
     }
+
+    /*
+    In the case where the customer passes a null BlobRange, constructing the default of "0-" will fail on an empty blob.
+    By returning null as the header value, we elect not to set the header, which has the same effect, namely downloading
+    the whole blob, but it will not fail in the empty case.
+     */
+    String toHeaderValue() {
+        // The default values of a BlobRange
+        if (this.offset == 0 && this.count == null) {
+            return null;
+        }
+        return this.toString();
+    }
 }