SQL core commands (server, db, and elastic pool) (Azure#2253)

* sql server create: param aliases

* sql db: server_name param alias

* sql db create: param aliases and ignores

* sql db data-warehouse: comment out these commands

* sql db transparent-data-encryption: comment out these commands

* sql db service-tier-advisor: comment out these commands

* sql db/server show-usage: comment out these commands

* sql elastic-pools: rename to elastic-pool

* sql elastic-pool recommended: comment out these commands

* sql db restore-point: comment out these commands

* sql db replication-link: -d param alias

* sql server firewall: add -s param alias

* sql server firewall allow-all-azure-ips: implement custom command

* Added CommandGroup.custom_command helper for constructing custom commands

* sql server service-objective: added -s alias

* sql elastic-pool db: Removed group. Added --elastic-pool parameter to sql db list.

* sql db update: Add convenience params

Fixes Azure#1764

* sql db create/update: extract common param aliases

* sql db create/update: refactor common ignored params

* sql db create: Wrapper function that determines location

* sql db create, sql elastic-pool db: fixed tests

* sql elastic-pool create: add wrapper that determines server location

* sql server update: add convenience params

* sql server firewall allow-all-azure-ips: added test

* Organized file contents into alphabetically ordered sections

* sql server firewall: updated help text

* sql elastic-pool update: add convenience arguments

* sql db create-copy: added this new custom command

* sql db create-copy: added unit test

* sql db create-replica: implement new custom command

* sql db copy/create-replica: switched point of view

It looks like source and dest sub id must always be the same, so removing source_subscription_id param and switching point of view from source->this to this->dest.

* sql db restore: added basic implementation

It's currently broken because restorePointInTime is not yet in Swagger spec, but otherwise should theoretically work.

* Tweaking param names for db copy/create-secondary/restore

* sql server firewall: rename group to firewall-rule

* Rename --server-name to --server

* Rename --elastic-pool-name to --elastic-pool

* Rename --database-name to --database

* Rename --service-objective-name to --service-objective

* Rename --admin-login to --admin-user

* Comment out db restore and firewll allow-all-azure-ips.

db restore is currently broken, and allow-all-azure-ips is hopefully not needed if the firewall create/update help text is clear enough (which I have updated in Swagger).

* Added missing help text and commented out unneeded help test

* Rename rg-name

* Add missing param help, and fixed dest/secondary param aliases

* sql db update: Added test and fixed bugs.

Other update commands will also need to be tested and updated because I misunderstood how generic update command works.

* az sql firewall-rule update: Found issues, commented out for now

* sql server firewall-rule update: implement as custom command

* Impl/test for elastic-pool update and db update --elastic-pool

* Fixed pylint errors

* Fixed flake8 errors

* Attempt to fix url quote import

* Ignore pylint no-name-in-module for url quote import

* Ignore pylint import-error for url quote import

* db copy: use dest server location, not source

* Standardized on replica terminology.

create-secondary is now create-replica. replication-link is now replica-link.

* sql db show: remove --expand param

* Doc improvements.

Removed redundant 'Azure SQL' terminology.
Added doc for sql db list --elastic-pool param.

* Minor doc tweaks

* sql db create: disallow creating datawarehouse

* Update azure-mgmt-sql dependency to 0.3.0

* Added sql db restore command.

Also updated params for other db creation commands:
- Ignore service_objective_id param for all.
- Updated create-mode specific params

* Remove and git ignore debug.log files

* service-objective list: don't verify response length.

The number of service-objectives returned is variable depending on feature light-up and subscription capabilities. It's best not to assume how many service objectives exist currently or in the future.

* sql elastic-pool list-dbs: new command

* Use 'six' compat library to import url quote

* Added unit converter and use it for storage params

* Corrected capitalization of byte unit (B)

.gitignore

@@ -70,6 +70,7 @@ command_coverage.txt
 # Test artifacts
 private_config.json
 scripts/smart_create_gen/config.ini
+debug.log
 test_results/
 resources/
 

src/command_modules/azure-cli-sql/azure/cli/command_modules/sql/_util.py

@@ -64,6 +64,7 @@ def __init__(self, scope, group_name, client_factory, service_adapter=None):
         self._group_name = group_name
         self._client_factory = client_factory
         self._service_adapter = service_adapter or (lambda name: name)
+        self._custom_path = 'azure.cli.command_modules.sql.custom#{}'
 
     def __enter__(self):
         return self
@@ -77,13 +78,25 @@ def command(self, name, method_name):
                     self._service_adapter(method_name),
                     client_factory=self._client_factory)
 
-    def generic_update_command(self, name, getter_op, setter_op):
+    def custom_command(self, name, custom_func_name):
+        cli_command(self._scope,
+                    '{} {}'.format(self._group_name, name),
+                    self._custom_path.format(custom_func_name),
+                    client_factory=self._client_factory)
+
+    def generic_update_command(self, name, getter_op, setter_op, custom_func_name=None):
+        if custom_func_name:
+            custom_function_op = self._custom_path.format(custom_func_name)
+        else:
+            custom_function_op = None
+
         cli_generic_update_command(
             self._scope,
             '{} {}'.format(self._group_name, name),
             self._service_adapter(getter_op),
             self._service_adapter(setter_op),
-            factory=self._client_factory)
+            factory=self._client_factory,
+            custom_function_op=custom_function_op)
 
 
 # PARAMETERS UTILITIES

src/command_modules/azure-cli-sql/azure/cli/command_modules/sql/commands.py

@@ -4,94 +4,119 @@
 # --------------------------------------------------------------------------------------------
 
 from ._util import (get_sql_servers_operation, get_sql_database_operations,
-                    get_sql_elasticpools_operations, get_sql_recommended_elastic_pools_operations,
-                    ServiceGroup, create_service_adapter)
+                    get_sql_elasticpools_operations, create_service_adapter, ServiceGroup)
 
-server_operations = create_service_adapter('azure.mgmt.sql.operations.servers_operations',
-                                           'ServersOperations')
-
-with ServiceGroup(__name__, get_sql_servers_operation, server_operations) as s:
-    with s.group('sql server') as c:
-        c.command('create', 'create_or_update')
-        c.command('delete', 'delete')
-        c.command('show', 'get_by_resource_group')
-        c.command('show-usage', 'list_usages')
-        c.command('list', 'list_by_resource_group')
-        c.generic_update_command('update', 'get_by_resource_group', 'create_or_update')
-
-    with s.group('sql server service-objective') as c:
-        c.command('list', 'list_service_objectives')
-        c.command('show', 'get_service_objective')
-
-    with s.group('sql server firewall') as c:
-        c.command('create', 'create_or_update_firewall_rule')
-        c.command('update', 'create_or_update_firewall_rule')
-        c.command('delete', 'delete_firewall_rule')
-        c.command('show', 'get_firewall_rule')
-        c.command('list', 'list_firewall_rules')
+###############################################
+#                sql db                       #
+###############################################
 
 database_operations = create_service_adapter('azure.mgmt.sql.operations.databases_operations',
                                              'DatabasesOperations')
 
 with ServiceGroup(__name__, get_sql_database_operations, database_operations) as s:
     with s.group('sql db') as c:
-        c.command('create', 'create_or_update')
+        c.custom_command('create', 'db_create')
+        c.custom_command('copy', 'db_copy')
+        c.custom_command('create-replica', 'db_create_replica')
+        c.custom_command('restore', 'db_restore')
         c.command('show', 'get')
-        c.command('list', 'list_by_server')
-        c.command('show-usage', 'list_usages')
+        c.custom_command('list', 'db_list')
+        # # Usages will not be included in the first batch of GA commands
+        # c.command('show-usage', 'list_usages')
         c.command('delete', 'delete')
-        c.generic_update_command('update', 'get', 'create_or_update')
+        c.generic_update_command('update', 'get', 'create_or_update', custom_func_name='db_update')
 
-    with s.group('sql db replication-link') as c:
+    with s.group('sql db replica-link') as c:
         c.command('list', 'list_replication_links')
         c.command('show', 'get_replication_link')
         c.command('delete', 'delete_replication_link')
         c.command('failover', 'failover_replication_link')
         c.command('force-failover', 'failover_replication_link_allow_data_loss')
 
-    with s.group('sql db data-warehouse') as c:
-        c.command('pause', 'pause_data_warehouse')
-        c.command('resume', 'resume_data_warehouse')
+    # Data Warehouse will not be included in the first batch of GA commands
+    # with s.group('sql db data-warehouse') as c:
+    #     c.command('pause', 'pause_data_warehouse')
+    #     c.command('resume', 'resume_data_warehouse')
 
-    with s.group('sql db restore-point') as c:
-        c.command('list', 'list_restore_points')
+    # Data Warehouse will not be included in the first batch of GA commands
+    # (list_restore_points also applies to db, but it's not very useful. It's
+    # mainly useful for dw.)
+    # with s.group('sql db restore-point') as c:
+    #     c.command('list', 'list_restore_points')
 
-    with s.group('sql db transparent-data-encryption') as c:
-        c.command('create', 'create_or_update_transparent_data_encryption_configuration')
-        c.command('show-configuration', 'get_transparent_data_encryption_configuration')
-        c.command('show-activity', 'list_transparent_data_encryption_activity')
+    # TDE will not be included in the first batch of GA commands
+    # with s.group('sql db transparent-data-encryption') as c:
+    #     c.command('create', 'create_or_update_transparent_data_encryption_configuration')
+    #     c.command('show-configuration', 'get_transparent_data_encryption_configuration')
+    #     c.command('show-activity', 'list_transparent_data_encryption_activity')
 
-    with s.group('sql db service-tier-advisor') as c:
-        c.command('list', 'list_service_tier_advisors')
-        c.command('show', 'get_service_tier_advisor')
+    # Service tier advisor will not be included in the first batch of GA commands
+    # with s.group('sql db service-tier-advisor') as c:
+    #     c.command('list', 'list_service_tier_advisors')
+    #     c.command('show', 'get_service_tier_advisor')
+
+###############################################
+#                sql elastic-pool             #
+###############################################
 
 elasticpools_ops = create_service_adapter('azure.mgmt.sql.operations.elastic_pools_operations',
                                           'ElasticPoolsOperations')
 
 with ServiceGroup(__name__, get_sql_elasticpools_operations, elasticpools_ops) as s:
-    with s.group('sql elastic-pools') as c:
-        c.command('create', 'create_or_update')
+    with s.group('sql elastic-pool') as c:
+        c.custom_command('create', 'elastic_pool_create')
         c.command('delete', 'delete')
         c.command('show', 'get')
         c.command('list', 'list_by_server')
-        c.generic_update_command('update', 'get', 'create_or_update')
-
-    with s.group('sql elastic-pools db') as c:
-        c.command('list', 'list_databases')
-        c.command('show', 'get_database')
-        c.command('show-activity', 'list_database_activity')
+        c.command('list-dbs', 'list_databases')
+        c.generic_update_command(
+            'update', 'get', 'create_or_update',
+            custom_func_name='elastic_pool_update')
 
 recommanded_elastic_pools_ops = \
     create_service_adapter('azure.mgmt.sql.operations.recommended_elastic_pools_operations',
                            'RecommendedElasticPoolsOperations')
 
-with ServiceGroup(__name__, get_sql_recommended_elastic_pools_operations,
-                  recommanded_elastic_pools_ops) as s:
-    with s.group('sql elastic-pools recommended') as c:
-        c.command('show', 'get')
-        c.command('show-metrics', 'list_metrics')
-        c.command('list', 'list')
+# Recommended elastic pools will not be included in the first batch of GA commands
+# with ServiceGroup(__name__, get_sql_recommended_elastic_pools_operations,
+#                   recommanded_elastic_pools_ops) as s:
+#    with s.group('sql elastic-pool recommended') as c:
+#        c.command('show', 'get')
+#        c.command('show-metrics', 'list_metrics')
+#        c.command('list', 'list')
+
+#    with s.group('sql elastic-pool recommended db') as c:
+#        c.command('show', 'get_databases')
+#        c.command('list', 'list_databases')
+
+###############################################
+#                sql server                   #
+###############################################
+
+server_operations = create_service_adapter('azure.mgmt.sql.operations.servers_operations',
+                                           'ServersOperations')
+
+with ServiceGroup(__name__, get_sql_servers_operation, server_operations) as s:
+    with s.group('sql server') as c:
+        c.command('create', 'create_or_update')
+        c.command('delete', 'delete')
+        c.command('show', 'get_by_resource_group')
+        # Usages will not be included in the first batch of GA commands
+        # c.command('show-usage', 'list_usages')
+        c.command('list', 'list_by_resource_group')
+        c.generic_update_command('update', 'get_by_resource_group', 'create_or_update',
+                                 custom_func_name='server_update')
 
-    with s.group('sql elastic-pools recommended db') as c:
-        c.command('show', 'get_databases')
-        c.command('list', 'list_databases')
+    with s.group('sql server service-objective') as c:
+        c.command('list', 'list_service_objectives')
+        c.command('show', 'get_service_objective')
+
+    with s.group('sql server firewall-rule') as c:
+        c.command('create', 'create_or_update_firewall_rule')
+        c.custom_command('update', 'firewall_rule_update')
+        c.command('delete', 'delete_firewall_rule')
+        c.command('show', 'get_firewall_rule')
+        c.command('list', 'list_firewall_rules')
+        # Keeping this command hidden for now. `firewall-rule create` will explain the special
+        # 0.0.0.0 rule.
+        # c.custom_command('allow-all-azure-ips', 'firewall_rule_allow_all_azure_ips')