-
Notifications
You must be signed in to change notification settings - Fork 4.1k
Personal/degoswami/newflags #29154
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Personal/degoswami/newflags #29154
Conversation
| Thanks for your contribution! The pull request validation has started. Please revisit this comment for updated status. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR extends the Az.Sftp module to make Connect-AzSftp more configurable and diagnosable by adding new connection options and enriching debug/verbose output across SFTP cmdlets.
Changes:
- Added
-BufferSize(int) to control thesftp -Btransfer buffer size. - Added
-StorageAccountEndpointto allow overriding the storage endpoint suffix used to build the SFTP hostname. - Expanded debug/verbose logging in
Connect-AzSftpandNew-AzSftpCertificate, and updated help + changelog + scenario tests accordingly.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| src/Sftp/Sftp/help/Connect-AzSftp.md | Documents new -BufferSize and -StorageAccountEndpoint parameters in syntax and parameter sections. |
| src/Sftp/Sftp/SftpCommands/ConnectAzSftpCommand.cs | Adds the two new parameters, passes buffer size into session, supports custom endpoint suffix, and adds detailed logging/cleanup messaging. |
| src/Sftp/Sftp/Models/SFTPSession.cs | Adds BufferSizeBytes and emits -B in the generated sftp command args (conditionally). |
| src/Sftp/Sftp/SftpCommands/NewAzSftpCertificateCommand.cs | Adds more structured debug/verbose logging and clearer warnings/errors around certificate generation. |
| src/Sftp/Sftp/CHANGELOG.md | Records the two new parameters and their intent under “Upcoming Release”. |
| src/Sftp/Sftp.Test/ScenarioTests/ConnectAzSftpTests.ps1 | Adds new scenario test functions intended to exercise the new parameters. |
| <# | ||
| .SYNOPSIS | ||
| Test Connect-AzSftp with BufferSize parameter | ||
| #> | ||
| function Test-ConnectAzSftpWithBufferSize | ||
| { | ||
| $storageAccountName = Get-StorageAccountName | ||
| $resourceGroupName = Get-ResourceGroupName | ||
|
|
||
| try { | ||
| # Skip test in playback mode for now | ||
| if (IsPlayback) { | ||
| return | ||
| } | ||
|
|
||
| # Create test storage account | ||
| $storageAccount = New-TestStorageAccount -ResourceGroupName $resourceGroupName -StorageAccountName $storageAccountName | ||
|
|
||
| # Test connection with custom buffer size (will fail in test environment but validates parameter parsing) | ||
| try { | ||
| $result = Connect-AzSftp -StorageAccount $storageAccountName -BufferSize 524288 -SftpArg "-o", "ConnectTimeout=1" | ||
| } | ||
| catch { | ||
| # Expected to fail in test environment - this is acceptable | ||
| Write-Host "Connection failed as expected in test environment: $($_.Exception.Message)" | ||
| } | ||
|
|
||
| # Test with default buffer size (256*1024 = 262144) | ||
| try { | ||
| $result = Connect-AzSftp -StorageAccount $storageAccountName -SftpArg "-o", "ConnectTimeout=1" | ||
| } | ||
| catch { | ||
| Write-Host "Connection failed as expected in test environment: $($_.Exception.Message)" | ||
| } | ||
| } | ||
| finally { | ||
| # Cleanup | ||
| Remove-TestStorageAccount -ResourceGroupName $resourceGroupName -StorageAccountName $storageAccountName | ||
| Remove-AzResourceGroup -Name $resourceGroupName -Force -ErrorAction SilentlyContinue | ||
| } | ||
| } | ||
|
|
||
| <# | ||
| .SYNOPSIS | ||
| Test Connect-AzSftp with StorageAccountEndpoint parameter | ||
| #> | ||
| function Test-ConnectAzSftpWithStorageAccountEndpoint | ||
| { | ||
| $storageAccountName = Get-StorageAccountName | ||
| $resourceGroupName = Get-ResourceGroupName | ||
| $customEndpoint = "blob.core.custom.endpoint.net" | ||
|
|
||
| try { | ||
| # Skip test in playback mode for now | ||
| if (IsPlayback) { | ||
| return | ||
| } | ||
|
|
||
| # Create test storage account | ||
| $storageAccount = New-TestStorageAccount -ResourceGroupName $resourceGroupName -StorageAccountName $storageAccountName | ||
|
|
||
| # Test connection with custom storage account endpoint (will fail in test environment but validates parameter parsing) | ||
| try { | ||
| $result = Connect-AzSftp -StorageAccount $storageAccountName -StorageAccountEndpoint $customEndpoint -SftpArg "-o", "ConnectTimeout=1" | ||
| } | ||
| catch { | ||
| # Expected to fail in test environment - this is acceptable | ||
| Write-Host "Connection failed as expected in test environment: $($_.Exception.Message)" | ||
| } | ||
| } | ||
| finally { | ||
| # Cleanup | ||
| Remove-TestStorageAccount -ResourceGroupName $resourceGroupName -StorageAccountName $storageAccountName | ||
| Remove-AzResourceGroup -Name $resourceGroupName -Force -ErrorAction SilentlyContinue | ||
| } | ||
| } |
Copilot
AI
Feb 9, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The added PowerShell scenario test functions don’t provide meaningful coverage in CI: the ConnectAzSftpTestsRunner has all [Fact] tests skipped, and these new functions also early-return in playback mode and only swallow connection failures without asserting on behavior (e.g., that -B is included in the generated sftp command, or that the custom endpoint affects the hostname). Add deterministic unit tests in the existing non-skipped C# test suite (e.g., SftpUtilsTests/SFTPSession tests) to validate -B argument emission and host construction, or otherwise ensure these tests are actually executed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like work.
Life's Always got its own Earth Shadow.
Keep is safe.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like work.
Life's Always got its own Earth Shadow.
Keep is safe.
src/Sftp/Sftp/Models/SFTPSession.cs
Outdated
| if (BufferSizeBytes != 256 * 1024) | ||
| { | ||
| args.AddRange(new[] { "-B", BufferSizeBytes.ToString() }); | ||
| } |
Copilot
AI
Feb 9, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
BufferSizeBytes is treated as if 256 KB is the default, but BuildArgs() only adds the -B flag when the value differs from 256*1024. As a result, when users do not pass -BufferSize (or pass 262144), the cmdlet won’t pass -B at all and the effective buffer size will fall back to the underlying sftp client’s default—contradicting the help/CHANGELOG that state a default of 262144. Decide on one behavior and make code+docs consistent (e.g., always pass -B with the default, or make buffer size optional/nullable and only add -B when the parameter was explicitly provided).
| if (BufferSizeBytes != 256 * 1024) | |
| { | |
| args.AddRange(new[] { "-B", BufferSizeBytes.ToString() }); | |
| } | |
| args.AddRange(new[] { "-B", BufferSizeBytes.ToString() }); |
| WriteDebug($"[SFTP] Waiting for SFTP session to complete before cleanup..."); | ||
| process.WaitForExit(); | ||
| // Wait up to 5 minutes (300,000 ms) for the process to exit | ||
| bool exited = process.WaitForExit(300000); | ||
| if (!exited) |
Copilot
AI
Feb 9, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The process wait logic makes the 5-minute timeout ineffective: process.WaitForExit() blocks indefinitely before the timed WaitForExit(300000) is called. This can hang the cmdlet and prevent credential cleanup. Remove the unconditional wait and rely on the timed wait (or apply a single wait strategy) so the timeout actually works.
|
That's a Healthy Conversation, |
|
@isra-fel Please review this PR |
|
/azp run |
|
Azure Pipelines successfully started running 3 pipeline(s). |
|
|
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
| throw new PSArgumentException( | ||
| "StorageAccountEndpoint must be a DNS suffix such as 'blob.core.windows.net' and must not contain a scheme or path."); | ||
| } | ||
|
|
||
| if (string.IsNullOrEmpty(normalizedSuffix)) | ||
| { | ||
| throw new PSArgumentException("StorageAccountEndpoint cannot be empty."); |
Copilot
AI
Feb 10, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The PSArgumentException thrown for invalid StorageAccountEndpoint doesn’t specify the parameter name, which makes PowerShell error output less actionable (it won’t highlight which argument is invalid). Consider using the overload that includes paramName (or an AzPSArgumentException with nameof(StorageAccountEndpoint)) for clearer diagnostics.
| throw new PSArgumentException( | |
| "StorageAccountEndpoint must be a DNS suffix such as 'blob.core.windows.net' and must not contain a scheme or path."); | |
| } | |
| if (string.IsNullOrEmpty(normalizedSuffix)) | |
| { | |
| throw new PSArgumentException("StorageAccountEndpoint cannot be empty."); | |
| throw new AzPSArgumentException( | |
| "StorageAccountEndpoint must be a DNS suffix such as 'blob.core.windows.net' and must not contain a scheme or path.", | |
| nameof(StorageAccountEndpoint)); | |
| } | |
| if (string.IsNullOrEmpty(normalizedSuffix)) | |
| { | |
| throw new AzPSArgumentException("StorageAccountEndpoint cannot be empty.", nameof(StorageAccountEndpoint)); |
src/Sftp/Sftp/help/Connect-AzSftp.md
Outdated
| ### -BufferSize | ||
| Buffer size in bytes for SFTP file transfers. Default: 262144 (256 KB). | ||
|
|
Copilot
AI
Feb 10, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The help text claims a default BufferSize of 262144 (256 KB), but the current implementation only adds the sftp -B flag when a non-default value is set. Unless -B 262144 is always emitted, the effective default will be whatever the sftp client uses. Please align the documentation with actual behavior (or update the code so the stated default is enforced).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Suggest naming it -BufferSizeInBytes for usability
| ## Upcoming Release | ||
| * Added `BufferSize` parameter to `Connect-AzSftp` cmdlet | ||
| - Allows users to specify buffer size in bytes for SFTP file transfers using the sftp -B flag | ||
| - Default value is 262144 (256 KB) |
Copilot
AI
Feb 10, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
CHANGELOG notes a BufferSize default of 262144 (256 KB), but the current sftp -B argument is only emitted when a non-default value is used. Either ensure the default is actually applied in the command invocation, or adjust the changelog entry to avoid stating an incorrect default behavior.
| - Default value is 262144 (256 KB) | |
| - When specified, a value of 262144 (256 KB) is recommended for optimal performance |
|
/azp run |
|
Azure Pipelines successfully started running 3 pipeline(s). |
|
@isra-fel please run the checks |
isra-fel
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please see the inline comments
src/Sftp/Sftp/help/Connect-AzSftp.md
Outdated
| ### -BufferSize | ||
| Buffer size in bytes for SFTP file transfers. Default: 262144 (256 KB). | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Suggest naming it -BufferSizeInBytes for usability
|
/azp run |
|
Azure Pipelines successfully started running 3 pipeline(s). |
|
This PR was labeled "needs-revision" because it has unresolved review comments or CI failures. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 9 out of 9 changed files in this pull request and generated 6 comments.
| // Arrange - default buffer size is 256 * 1024 = 262144 | ||
| var session = new SFTPSession( | ||
| storageAccount: "testaccount", | ||
| username: "testuser", | ||
| host: "testaccount.blob.core.windows.net", | ||
| port: 22, | ||
| publicKeyFile: null, | ||
| privateKeyFile: null, | ||
| certFile: null, | ||
| sftpArgs: null, | ||
| sshClientFolder: null, | ||
| sshProxyFolder: null, | ||
| credentialsFolder: null, | ||
| yesWithoutPrompt: false, | ||
| bufferSizeBytes: 256 * 1024 // Default value | ||
| ); |
Copilot
AI
Feb 10, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These tests hardcode the default buffer size as 256 * 1024 / 262144. Since the production default is defined in SftpConstants.DefaultBufferSizeBytes, using the constant here would keep tests aligned if the default ever changes and avoid duplicating magic numbers.
| PublicKeyFile, PrivateKeyFile, credentialsFolder, SshClientFolder); | ||
| PublicKeyFile = pubKey; | ||
| PrivateKeyFile = privKey; | ||
|
|
Copilot
AI
Feb 10, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FileUtils.CheckOrCreatePublicPrivateFiles returns a deleteKeys flag (delKeys here) indicating whether key material was generated and should be cleaned up, but the value is never applied to the cmdlet-level deleteKeys. This can leave newly generated keys on disk unexpectedly (and they contain sensitive material). Propagate delKeys into deleteKeys (e.g., deleteKeys |= delKeys) so cleanup behavior matches what FileUtils reports.
| deleteKeys |= delKeys; |
| // Use custom endpoint suffix if provided | ||
| if (!string.IsNullOrWhiteSpace(StorageAccountEndpoint)) | ||
| { | ||
| var normalizedSuffix = StorageAccountEndpoint.Trim(); | ||
|
|
||
| // Remove any leading dots so that values like ".blob.core.windows.net" are accepted | ||
| normalizedSuffix = normalizedSuffix.TrimStart('.'); | ||
|
|
||
| // Reject values that look like full URLs or contain paths, since we only expect a DNS suffix | ||
| if (normalizedSuffix.IndexOf("://", StringComparison.Ordinal) >= 0 || | ||
| normalizedSuffix.IndexOf("/", StringComparison.Ordinal) >= 0) | ||
| { | ||
| throw new PSArgumentException( | ||
| "StorageAccountEndpoint must be a DNS suffix such as 'blob.core.windows.net' and must not contain a scheme or path."); | ||
| } | ||
|
|
||
| if (string.IsNullOrEmpty(normalizedSuffix)) | ||
| { | ||
| throw new PSArgumentException("StorageAccountEndpoint cannot be empty."); | ||
| } | ||
|
|
||
| return normalizedSuffix; | ||
| } |
Copilot
AI
Feb 10, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
StorageAccountEndpoint introduces new parsing/validation behavior (trimming leading dots and rejecting schemes/paths), but there are no unit tests covering valid/invalid inputs or verifying the computed hostname uses the custom suffix. Add deterministic C# tests for GetStorageEndpointSuffix behavior (or refactor it into a testable helper) so regressions are caught without relying on skipped E2E scripts.
| // Test that BufferSizeInBytes has correct default value (256 * 1024 = 262144) | ||
| var connectCommand = new ConnectAzSftpCommand(); | ||
|
|
||
| Assert.Equal(256 * 1024, connectCommand.BufferSizeInBytes); | ||
| Assert.Equal(262144, connectCommand.BufferSizeInBytes); |
Copilot
AI
Feb 10, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The default-value test duplicates the production constant as 256 * 1024 and 262144. Prefer asserting against SftpConstants.DefaultBufferSizeBytes so tests remain correct if the default is ever adjusted, and you only have one source of truth.
| // Test that BufferSizeInBytes has correct default value (256 * 1024 = 262144) | |
| var connectCommand = new ConnectAzSftpCommand(); | |
| Assert.Equal(256 * 1024, connectCommand.BufferSizeInBytes); | |
| Assert.Equal(262144, connectCommand.BufferSizeInBytes); | |
| // Test that BufferSizeInBytes has correct default value | |
| var connectCommand = new ConnectAzSftpCommand(); | |
| Assert.Equal(SftpConstants.DefaultBufferSizeBytes, connectCommand.BufferSizeInBytes); |
| return normalizedSuffix; | ||
| } | ||
|
|
||
| string cloudName = DefaultContext?.Environment?.Name?.ToLower() ?? "azurecloud"; |
Copilot
AI
Feb 10, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
DefaultContext?.Environment?.Name?.ToLower() uses current-culture casing rules. For environment name comparisons, use ToLowerInvariant() to avoid culture-specific edge cases (e.g., Turkish-I).
| string cloudName = DefaultContext?.Environment?.Name?.ToLower() ?? "azurecloud"; | |
| string cloudName = DefaultContext?.Environment?.Name?.ToLowerInvariant() ?? "azurecloud"; |
| // Reject values that look like full URLs or contain paths, since we only expect a DNS suffix | ||
| if (normalizedSuffix.IndexOf("://", StringComparison.Ordinal) >= 0 || | ||
| normalizedSuffix.IndexOf("/", StringComparison.Ordinal) >= 0) | ||
| { | ||
| throw new PSArgumentException( | ||
| "StorageAccountEndpoint must be a DNS suffix such as 'blob.core.windows.net' and must not contain a scheme or path."); | ||
| } |
Copilot
AI
Feb 10, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
StorageAccountEndpoint is treated as a DNS suffix, but the current validation allows characters like :. In OpenSSH sftp, a destination of the form host:dir is parsed as an initial remote path, so allowing : in the suffix can change semantics unexpectedly. Consider rejecting : (and other non-DNS characters/whitespace) so the parameter is constrained to a true hostname suffix.
|
/azp run |
|
Azure Pipelines successfully started running 3 pipeline(s). |
isra-fel
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
Description
Mandatory Checklist
Please choose the target release of Azure PowerShell. (⚠️ Target release is a different concept from API readiness. Please click below links for details.)
Check this box to confirm: I have read the Submitting Changes section of
CONTRIBUTING.mdand reviewed the following information:ChangeLog.mdfile(s) appropriatelysrc/{{SERVICE}}/{{SERVICE}}/ChangeLog.md.## Upcoming Releaseheader in the past tense.ChangeLog.mdif no new release is required, such as fixing test case only.