Skip to content

[SQL] Add short-term backup retention immutability support - #34207

Draft
svmicano wants to merge 2 commits into
Azure:devfrom
svmicano:svmicano/sql-str-immutability
Draft

svmicano wants to merge 2 commits into
Azure:devfrom
svmicano:svmicano/sql-str-immutability

Conversation

@svmicano

@svmicano svmicano commented Oct 9, 2026 •

Copy link
Copy Markdown
Member

Related command

  • az sql db str-policy set, show, and wait
  • az sql midb short-term-retention-policy set and show (live and restorable dropped databases)

Description
Add short-term backup immutability support using 2026-08-01-preview, matching the behavior in Azure/azure-powershell#30215.

  • Add nullable --lock-immutability and --yes/-y to both set commands. Only true prompts, with an irreversible-action warning. Cancellation sends no update and emits no result; non-interactive callers must use --yes when locking.
  • Reject --lock-immutability together with --deleted-time, including explicit false, and never include that property in a dropped MI database request.
  • Surface the service's immutabilityStatus on set/show for SQL Database and live/dropped MI databases while preserving flattened CLI output.
  • Allow SQL Database lock-only/differential-interval-only changes without sending omitted settings as null. Preserve required retention days for MI and zero retention for dropped MI databases.
  • Use shared SDK-independent AAZ HTTP/polling operations behind the existing command registrations. Preserve URL escaping, cloud endpoints, authentication, --ids, SQL DB wait, and --no-wait behavior.
  • Keep azure-mgmt-sql==4.0.0b22 unchanged. The referenced https://pypi.org/project/azure-mgmt-sql/5.0.0b2/ introduces hybrid models and unrelated breaking changes; a whole-module SDK migration is deliberately out of scope.
  • Update help and add 22 offline CLI/HTTP tests, four MI scenario safety/polling tests, and two live-recorded MI scenarios covering ordinary and immutable live/dropped STR behavior.

Draft / before marking ready

  • Re-record test_sql_managed_db_short_retention against 2026-08-01-preview and add test_sql_managed_db_short_retention_immutability; both MI scenarios passed live and in offline playback.
  • Validate MI locking, confirmation, --yes, live/dropped policy reads, and service rejection of unlocking and retention reductions. Test-created databases were cleaned up; MI configuration and all seven pre-existing databases were unchanged.
  • Re-record test_sql_db_short_term_retention and complete SQL Database (non-MI) live validation. Its old API recording has not been refreshed.
  • Run full SQL lint/scenario validation in the standard development environment. Full-module local Pylint could not resolve compiled-only dependencies in the installed CLI runtime and also reported existing findings; it is not claimed as passing.

Known backend response discrepancy
The MI operation-result endpoint can report immutabilityStatus: Disabled after successful locking while the database policy GET reports Locked. A separate diagnostic observed this mismatch across 24 paired reads over approximately 144 seconds. Per the agreed scope, locking tests poll only the database policy GET every five seconds (300-second timeout), then assert status and retention through show. They do not assert immutability status on the SET operation-result response. This backend issue is being investigated separately; no fixed delay or readback workaround was added to the CLI implementation.

Testing Guide
Validated with edited CLI/core source and installed Azure CLI libraries (Python 3.14.6):

  • 28 tests passed: 22 CLI/HTTP unit tests, four MI scenario safety/polling tests, and two recorded MI scenarios. Offline verification used an empty Azure configuration, no live-target environment variables, blocked socket connections, and a disabled unrelated release-update connectivity probe.
  • Both MI scenarios passed live on an existing feature-enabled production test MI in West Central US. The locking scenario passed confirmation acceptance, --yes, locked readback, retention increase, rejection of retention decrease and unlock, and locked dropped-database validation.
  • The recordings retain actual operation-result responses (including the backend discrepancy); recorded response statuses were not rewritten to make tests pass.
  • Recording audit passed: subscription/resource identifiers and signed polling query values are sanitized; unrelated dropped databases are removed; recorded writes are confined to the scenario's own database and STR policy.
  • All five affected command help pages execute successfully (covered by unit tests).
  • Pylint for the new STR operation module previously passed at 10.00/10; that product module is unchanged by the test update.
  • pycodestyle for the updated test modules and git diff --check: passed.
  • SQL Database scenario recording refresh and full SQL-module validation remain outstanding.

To replay in a configured developer checkout:
python -m unittest azure.cli.command_modules.sql.tests.latest.test_sql_str_policy azure.cli.command_modules.sql.tests.latest.test_sql_mi_str_policy

For live recording, see src/azure-cli/azure/cli/command_modules/sql/tests/latest/README.md. An explicit AZURE_CLI_TEST_SQL_MI_RESOURCE_ID is required; irreversible locking additionally requires AZURE_CLI_TEST_SQL_MI_ALLOW_IMMUTABILITY_LOCK=true. These scenarios never create, modify or delete the MI or resource group; they create and clean up only randomized databases. Do not run the entire SQL suite live against a shared instance.

Earlier West Europe smoke results used a target where locking was unavailable. The later successful feature-enabled MI recordings supersede that MI locking blocker. Locked test backups remain under their configured retention; they were not purged.

History Notes
[SQL] az sql db str-policy: Add short-term backup immutability locking with confirmation and expose immutability status
[SQL] az sql midb short-term-retention-policy: Add immutability locking for live databases and expose status for live and dropped databases


This checklist is used to make sure that common guidelines for a pull request are followed.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).

@yonzhan

Copy link
Copy Markdown
Collaborator

Please fix CI issues

@svmicano

Copy link
Copy Markdown
Member Author

Production MI validation — 2026-10-12

Validated commit 3fe125128615c9c7a7958520ccb10a970fa9be22 by running the edited CLI/core source against the public Azure production ARM endpoint in West Europe. Two new, uniquely named databases were created on an existing test MI. No MI-level write requests were made, and pre-existing databases were not modified.

Check Result
Preview STR show for both live databases Passed; retentionDays: 7, immutabilityStatus: Disabled initially
Synchronous retention update with lock flag omitted Passed on both databases: 7 → 14 days, verified by show
Explicit --lock-immutability false without --yes Passed without prompting: 14 → 7 days
Live --no-wait Passed: update returned without a result, then read-only polling/show confirmed 10 days
Declined confirmation / no-TTY safeguards Passed using a controlled prompt host: exit 1, no result, zero HTTP requests; live readback was unchanged
Authorized --lock-immutability true --yes Service rejected with LockPitrImmutabilityNotAllowed; readback remained Disabled, with retention unchanged
Dropped database STR show Passed after confirming backup readiness and dropping only a new test database
Dropped STR synchronous update Passed: 10 → 7 days, verified by show
Dropped STR --no-wait Passed: 7 → 6 days, verified by read-only polling/show
Dropped --lock-immutability true and false Both rejected locally: exit 1, no result, zero HTTP requests, no prompt
Cleanup / scope safety Both newly created databases deleted; MI configuration matched the initial baseline, and the two original databases remained online

The dropped test backups remain under their configured retention; they were not purged. Use the original textual deletionDate from az sql midb list-deleted (for example with --query ... --output tsv) so shell date conversion does not lose fractional seconds.

Still blocked: this MI reports immutability Disabled, and the service explicitly disallows locking. Successful locking, Locked readback and rejection of retention decreases after locking therefore remain unvalidated on live resources. No MI configuration was changed to work around this limitation. SQL Database (non-MI) live validation was not performed in this run.

These were live smoke checks, not newly captured/re-recorded scenario cassettes. Existing recording refresh and standard full-module validation remain outstanding; keep this PR in draft. No implementation changes were required by the supported-path checks.

@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants