Description
openedon May 14, 2020
Describe the bug
For methods taking in a Uri as a parameter and later being used as a header value, in attempt to convert the Uri into a string, it unencodes the Uri before it sets it as the header value.
By calling Uri.ToString(), it automatically unencodes the header value.
This is causing bugs in the storage sdk when the Uri is being used. For example in the StageBlockFromUri method we pass a sourceUri, but in the generated code it will convert the Uri into a string, thus unencoding it unintentionally.
Expected behavior
The generated code should be calling Uri.AbsoluteUri() instead of Uri.ToString() in order to retrieve an unencoded Uri, or recognize when to get the absolute/encoded/escape character URL.
Actual behavior (include Exception or Stack Trace)
Anytime a customer calls one of our methods with a URL that has non ascii characters, they will get this exception.
System.AggregateException : Retry failed after 6 tries. (Request headers must contain only ASCII characters.) (Request headers must contain only ASCII characters.) (Request headers must contain only ASCII characters.) (Request headers must contain only ASCII characters.) (Request headers must contain only ASCII characters.) (Request headers must contain only ASCII characters.)
----> Azure.RequestFailedException : Request headers must contain only ASCII characters.
----> Azure.RequestFailedException : Request headers must contain only ASCII characters.
----> Azure.RequestFailedException : Request headers must contain only ASCII characters.
----> Azure.RequestFailedException : Request headers must contain only ASCII characters.
----> Azure.RequestFailedException : Request headers must contain only ASCII characters.
----> Azure.RequestFailedException : Request headers must contain only ASCII characters.
----> System.Net.Http.HttpRequestException : Request headers must contain only ASCII characters.
----> System.Net.Http.HttpRequestException : Request headers must contain only ASCII characters.
----> System.Net.Http.HttpRequestException : Request headers must contain only ASCII characters.
----> System.Net.Http.HttpRequestException : Request headers must contain only ASCII characters.
----> System.Net.Http.HttpRequestException : Request headers must contain only ASCII characters.
----> System.Net.Http.HttpRequestException : Request headers must contain only ASCII characters.
Stack Trace:
RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
RetryPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipeline.Send(HttpMessage message, CancellationToken cancellationToken)
BlockBlob.StageBlockFromUriAsync(ClientDiagnostics clientDiagnostics, HttpPipeline pipeline, Uri resourceUri, String blockId, Int64 contentLength, Uri sourceUri, String version, String sourceRange, Byte[] sourceContentHash, Byte[] sourceContentcrc64, Nullable`1 timeout, String encryptionKey, String encryptionKeySha256, Nullable`1 encryptionAlgorithm, String encryptionScope, String leaseId, Nullable`1 sourceIfModifiedSince, Nullable`1 sourceIfUnmodifiedSince, Nullable`1 sourceIfMatch, Nullable`1 sourceIfNoneMatch, String requestId, Boolean async, String operationName, CancellationToken cancellationToken) line 11405
BlockBlobClient.StageBlockFromUriInternal(Uri sourceUri, String base64BlockId, HttpRange sourceRange, Byte[] sourceContentHash, RequestConditions sourceConditions, BlobRequestConditions conditions, Boolean async, CancellationToken cancellationToken) line 1056
BlockBlobClient.StageBlockFromUri(Uri sourceUri, String base64BlockId, HttpRange sourceRange, Byte[] sourceContentHash, RequestConditions sourceConditions, BlobRequestConditions conditions, CancellationToken cancellationToken) line 881
--- End of stack trace from previous location where exception was thrown ---
<<StageBlockFromUriAsync_nonAsciiUri>b__0>d.MoveNext() line 708
--- End of stack trace from previous location where exception was thrown ---
StorageTestBase.RetryAsync[T](RecordedTestMode mode, Func`1 operation, Func`2 shouldRetry, Int32 retryDelay, Int32 retryAttempts)
StorageTestBase.RetryAsync[T](Func`1 operation, Func`2 shouldRetry, Int32 retryDelay, Int32 retryAttempts)
BlockBlobClientTests.StageBlockFromUriAsync_nonAsciiUri() line 707
BlockBlobClientTests.StageBlockFromUriAsync_nonAsciiUri() line 707
GenericAdapter`1.GetResult() line 99
AsyncToSyncAdapter.Await(Func`1 invoke) line 60
TestMethodCommand.Execute(TestExecutionContext context) line 64
<>c__DisplayClass1_0.<Execute>b__0() line 58
BeforeAndAfterTestCommand.RunTestMethodInThreadAbortSafeZone(TestExecutionContext context, Action action) line 73
--RequestFailedException
HttpClientTransport.ProcessAsync(HttpMessage message)
HttpClientTransport.Process(HttpMessage message)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
RequestActivityPolicy.ProcessNextAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean isAsync)
RequestActivityPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean isAsync)
RequestActivityPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
ResponseBodyPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
ResponseBodyPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
LoggingPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
LoggingPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
--RequestFailedException
HttpClientTransport.ProcessAsync(HttpMessage message)
HttpClientTransport.Process(HttpMessage message)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
RequestActivityPolicy.ProcessNextAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean isAsync)
RequestActivityPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean isAsync)
RequestActivityPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
ResponseBodyPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
ResponseBodyPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
LoggingPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
LoggingPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
--RequestFailedException
HttpClientTransport.ProcessAsync(HttpMessage message)
HttpClientTransport.Process(HttpMessage message)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
RequestActivityPolicy.ProcessNextAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean isAsync)
RequestActivityPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean isAsync)
RequestActivityPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
ResponseBodyPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
ResponseBodyPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
LoggingPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
LoggingPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
--RequestFailedException
HttpClientTransport.ProcessAsync(HttpMessage message)
HttpClientTransport.Process(HttpMessage message)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
RequestActivityPolicy.ProcessNextAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean isAsync)
RequestActivityPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean isAsync)
RequestActivityPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
ResponseBodyPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
ResponseBodyPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
LoggingPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
LoggingPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
--RequestFailedException
HttpClientTransport.ProcessAsync(HttpMessage message)
HttpClientTransport.Process(HttpMessage message)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
RequestActivityPolicy.ProcessNextAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean isAsync)
RequestActivityPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean isAsync)
RequestActivityPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
ResponseBodyPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
ResponseBodyPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
LoggingPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
LoggingPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
--RequestFailedException
HttpClientTransport.ProcessAsync(HttpMessage message)
HttpClientTransport.Process(HttpMessage message)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
RequestActivityPolicy.ProcessNextAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean isAsync)
RequestActivityPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean isAsync)
RequestActivityPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
ResponseBodyPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
ResponseBodyPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
LoggingPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
LoggingPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
--HttpRequestException
HttpConnection.WriteStringAsync(String s)
HttpConnection.WriteHeadersAsync(HttpHeaders headers, String cookiesFromContainer)
HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
DiagnosticsHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
HttpClientTransport.ProcessAsync(HttpMessage message)
--HttpRequestException
HttpConnection.WriteStringAsync(String s)
HttpConnection.WriteHeadersAsync(HttpHeaders headers, String cookiesFromContainer)
HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
DiagnosticsHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
HttpClientTransport.ProcessAsync(HttpMessage message)
--HttpRequestException
HttpConnection.WriteStringAsync(String s)
HttpConnection.WriteHeadersAsync(HttpHeaders headers, String cookiesFromContainer)
HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
DiagnosticsHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
HttpClientTransport.ProcessAsync(HttpMessage message)
--HttpRequestException
HttpConnection.WriteStringAsync(String s)
HttpConnection.WriteHeadersAsync(HttpHeaders headers, String cookiesFromContainer)
HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
DiagnosticsHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
HttpClientTransport.ProcessAsync(HttpMessage message)
--HttpRequestException
HttpConnection.WriteStringAsync(String s)
HttpConnection.WriteHeadersAsync(HttpHeaders headers, String cookiesFromContainer)
HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
DiagnosticsHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
HttpClientTransport.ProcessAsync(HttpMessage message)
--HttpRequestException
HttpConnection.WriteStringAsync(String s)
HttpConnection.WriteHeadersAsync(HttpHeaders headers, String cookiesFromContainer)
HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
DiagnosticsHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
HttpClientTransport.ProcessAsync(HttpMessage message)
To Reproduce
Refer to issue #11368
[Test]
public async Task StageBlockFromUriAsync_nonAsciiUri()
{
await using DisposingContainer test = await GetTestContainerAsync();
// Arrange
const int blobSize = Constants.KB;
var data = GetRandomBuffer(blobSize);
BlockBlobClient sourceBlob = InstrumentClient(test.Container.GetBlockBlobClient("sample-blob‽"));
using (var stream = new MemoryStream(data))
{
await sourceBlob.UploadAsync(stream);
}
BlockBlobClient destBlob = InstrumentClient(test.Container.GetBlockBlobClient(GetNewBlobName()));
// Act
await RetryAsync(
async () => await destBlob.StageBlockFromUriAsync(sourceBlob.Uri, ToBase64(GetNewBlockName())),
_retryStageBlockFromUri);
}