[purview] add catalog client (Azure#17788)

sdk/purview/azure-purview-catalog/CHANGELOG.md

@@ -0,0 +1,5 @@
+# Release History
+
+## 1.0.0b1 (2021-05-11)
+
+- This is the initial release of the Azure Purview Catalog library.

sdk/purview/azure-purview-catalog/MANIFEST.in

@@ -0,0 +1,5 @@
+recursive-include tests *.py
+recursive-include samples *.py *.md
+include *.md
+include azure/__init__.py
+include azure/purview/__init__.py

sdk/purview/azure-purview-catalog/README.md

@@ -0,0 +1,170 @@
+# Azure Purview Catalog client library for Python
+
+Azure Purview Catalog is a fully managed cloud service whose users can discover the data sources they need and understand the data sources they find. At the same time, Data Catalog helps organizations get more value from their existing investments.
+
+- Search for data using technical or business terms
+- Browse associated technical, business, semantic, and operational metadata
+- Identify the sensitivity level of data.
+
+**Please rely heavily on the [service's documentation][catalog_product_documentation] and our [client docs][request_builders_and_client] to use this library**
+
+[Source code][source_code] | [Package (PyPI)][catalog_pypi] | [API reference documentation][catalog_ref_docs]| [Product documentation][catalog_product_documentation]
+
+## Getting started
+
+### Prerequisites
+
+- Python 2.7, or 3.6 or later is required to use this package.
+- You must have an [Azure subscription][azure_subscription] and a [Purview][purview_resource] to use this package.
+
+#### Create a Purview Resource
+
+Follow [these][purview_resource] instructions to create your Purview resource
+
+### Install the package
+
+Install the Azure Purview Catalog client library for Python with [pip][pip]:
+
+```bash
+pip install azure-purview-catalog
+```
+
+### Authenticate the client
+
+To use an [Azure Active Directory (AAD) token credential][authenticate_with_token],
+provide an instance of the desired credential type obtained from the
+[azure-identity][azure_identity_credentials] library.
+
+To authenticate with AAD, you must first [pip][pip] install [`azure-identity`][azure_identity_pip] and
+[enable AAD authentication on your Purview resource][enable_aad]
+
+After setup, you can choose which type of [credential][azure_identity_credentials] from azure.identity to use.
+As an example, [DefaultAzureCredential][default_azure_credential]
+can be used to authenticate the client:
+
+Set the values of the client ID, tenant ID, and client secret of the AAD application as environment variables:
+AZURE_CLIENT_ID, AZURE_TENANT_ID, AZURE_CLIENT_SECRET
+
+Use the returned token credential to authenticate the client:
+
+```python
+from azure.purview.catalog import PurviewCatalogClient
+from azure.identity import DefaultAzureCredential
+
+credential = DefaultAzureCredential()
+client = PurviewCatalogClient(endpoint="https://<my-account-name>.catalog.purview.azure.com", credential=credential)
+```
+
+## Key concepts
+
+### Client
+
+This package offers request builders so you can build http requests and send these requests to the service using the `send_request` method.
+For more information on how to use request builders and our clients, see [here][request_builders_and_client].
+
+## Examples
+
+The following section shows you how to initialize and authenticate your client, then get all of your type-defs.
+
+- [Get All Type Definitions](#get-all-type-definitions "Get All Type Definitions")
+
+### Get All Type Definitions
+
+```python
+from azure.purview.catalog import PurviewCatalogClient
+from azure.identity import DefaultAzureCredential
+from azure.purview.catalog.rest import types
+from azure.core.exceptions import HttpResponseError
+
+credential = DefaultAzureCredential()
+client = PurviewCatalogClient(endpoint="https://<my-account-name>.catalog.purview.azure.com", credential=credential)
+
+request = types.build_get_all_type_definitions_request()
+
+response = client.send_request(request)
+try:
+    response.raise_for_status()
+    json_response = response.json()
+    # print out all of your entity definitions
+    print(json_response['entityDefs'])
+
+except HttpResponseError as e:
+    print(e)
+```
+
+## Troubleshooting
+
+### General
+
+The Purview Catalog client will raise exceptions defined in [Azure Core][azure_core] if you call `.raise_for_status()` on your responses.
+
+### Logging
+
+This library uses the standard
+[logging][python_logging] library for logging.
+Basic information about HTTP sessions (URLs, headers, etc.) is logged at INFO
+level.
+
+Detailed DEBUG level logging, including request/response bodies and unredacted
+headers, can be enabled on a client with the `logging_enable` keyword argument:
+
+```python
+import sys
+import logging
+from azure.identity import DefaultAzureCredential
+from azure.purview.catalog import PurviewCatalogClient
+
+# Create a logger for the 'azure' SDK
+logger = logging.getLogger('azure')
+logger.setLevel(logging.DEBUG)
+
+# Configure a console output
+handler = logging.StreamHandler(stream=sys.stdout)
+logger.addHandler(handler)
+
+endpoint = "https://<my-account-name>.catalog.purview.azure.com"
+credential = DefaultAzureCredential()
+
+# This client will log detailed information about its HTTP sessions, at DEBUG level
+client = PurviewCatalogClient(endpoint=endpoint, credential=credential, logging_enable=True)
+```
+
+Similarly, `logging_enable` can enable detailed logging for a single `send_request` call,
+even when it isn't enabled for the client:
+
+```python
+result = client.send_request(request, logging_enable=True)
+```
+
+## Next steps
+
+For more generic samples, see our [client docs][request_builders_and_client].
+
+## Contributing
+
+This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit [cla.microsoft.com][cla].
+
+When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
+
+This project has adopted the [Microsoft Open Source Code of Conduct][code_of_conduct]. For more information see the [Code of Conduct FAQ][coc_faq] or contact [opencode@microsoft.com][coc_contact] with any additional questions or comments.
+
+<!-- LINKS -->
+
+[source_code]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/purview/azure-purview-catalog/azure/purview/catalog
+[catalog_pypi]: https://aka.ms/azsdk/python/purviewcatalog/pypi
+[catalog_ref_docs]: https://aka.ms/azsdk/python/purviewcatalog/ref-docs
+[catalog_product_documentation]: https://azure.microsoft.com/services/purview/
+[azure_subscription]: https://azure.microsoft.com/free/
+[purview_resource]: https://docs.microsoft.com/azure/purview/create-catalog-portal
+[pip]: https://pypi.org/project/pip/
+[authenticate_with_token]: https://docs.microsoft.com/azure/cognitive-services/authentication?tabs=powershell#authenticate-with-an-authentication-token
+[azure_identity_credentials]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/identity/azure-identity#credentials
+[azure_identity_pip]: https://pypi.org/project/azure-identity/
+[default_azure_credential]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/identity/azure-identity#defaultazurecredential
+[request_builders_and_client]: https://aka.ms/azsdk/python/protocol/quickstart
+[enable_aad]: https://docs.microsoft.com/azure/purview/create-catalog-portal#add-a-security-principal-to-a-data-plane-role
+[python_logging]: https://docs.python.org/3.5/library/logging.html
+[cla]: https://cla.microsoft.com
+[code_of_conduct]: https://opensource.microsoft.com/codeofconduct/
+[coc_faq]: https://opensource.microsoft.com/codeofconduct/faq/
+[coc_contact]: mailto:opencode@microsoft.com

sdk/purview/azure-purview-catalog/azure/__init__.py

@@ -0,0 +1 @@
+__path__ = __import__('pkgutil').extend_path(__path__, __name__)  # type: ignore

sdk/purview/azure-purview-catalog/azure/purview/__init__.py

@@ -0,0 +1 @@
+__path__ = __import__('pkgutil').extend_path(__path__, __name__)  # type: ignore

sdk/purview/azure-purview-catalog/azure/purview/catalog/__init__.py

@@ -0,0 +1,19 @@
+# coding=utf-8
+# --------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License. See License.txt in the project root for license information.
+# Code generated by Microsoft (R) AutoRest Code Generator.
+# Changes may cause incorrect behavior and will be lost if the code is regenerated.
+# --------------------------------------------------------------------------
+
+from ._azure_purview_catalog_client import PurviewCatalogClient
+from ._version import VERSION
+
+__version__ = VERSION
+__all__ = ['PurviewCatalogClient']
+
+try:
+    from ._patch import patch_sdk  # type: ignore
+    patch_sdk()
+except ImportError:
+    pass

sdk/purview/azure-purview-catalog/azure/purview/catalog/_azure_purview_catalog_client.py

@@ -0,0 +1,104 @@
+# coding=utf-8
+# --------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License. See License.txt in the project root for license information.
+# Code generated by Microsoft (R) AutoRest Code Generator.
+# Changes may cause incorrect behavior and will be lost if the code is regenerated.
+# --------------------------------------------------------------------------
+
+from copy import deepcopy
+from typing import TYPE_CHECKING
+
+from azure.core import PipelineClient
+from azure.purview.catalog.core.rest import HttpResponse, _StreamContextManager
+from msrest import Deserializer, Serializer
+
+if TYPE_CHECKING:
+    # pylint: disable=unused-import,ungrouped-imports
+    from typing import Any, Dict
+
+    from azure.core.credentials import TokenCredential
+    from azure.purview.catalog.core.rest import HttpRequest
+
+from ._configuration import PurviewCatalogClientConfiguration
+
+
+class PurviewCatalogClient(object):
+    """Purview Catalog Service is a fully managed cloud service whose users can discover the data sources they need and understand the data sources they find. At the same time, Data Catalog helps organizations get more value from their existing investments. This spec defines REST API of Purview Catalog Service.
+
+    :param credential: Credential needed for the client to connect to Azure.
+    :type credential: ~azure.core.credentials.TokenCredential
+    :param endpoint: The catalog endpoint of your Purview account. Example: https://{accountName}.catalog.purview.azure.com.
+    :type endpoint: str
+    """
+
+    def __init__(
+        self,
+        credential,  # type: "TokenCredential"
+        endpoint,  # type: str
+        **kwargs  # type: Any
+    ):
+        # type: (...) -> None
+        base_url = '{Endpoint}/api'
+        self._config = PurviewCatalogClientConfiguration(credential, endpoint, **kwargs)
+        self._client = PipelineClient(base_url=base_url, config=self._config, **kwargs)
+
+        self._serialize = Serializer()
+        self._deserialize = Deserializer()
+        self._serialize.client_side_validation = False
+
+    def send_request(self, http_request, **kwargs):
+        # type: (HttpRequest, Any) -> HttpResponse
+        """Runs the network request through the client's chained policies.
+
+        We have helper methods to create requests specific to this service in `azure.purview.catalog.rest`.
+        Use these helper methods to create the request you pass to this method. See our example below:
+
+        >>> from azure.purview.catalog.rest import build_create_or_update_request
+        >>> request = build_create_or_update_request(json, content)
+        <HttpRequest [POST], url: '/atlas/v2/entity'>
+        >>> response = client.send_request(request)
+        <HttpResponse: 200 OK>
+
+        For more information on this code flow, see https://aka.ms/azsdk/python/protocol/quickstart
+
+        For advanced cases, you can also create your own :class:`~azure.purview.catalog.core.rest.HttpRequest`
+        and pass it in.
+
+        :param http_request: The network request you want to make. Required.
+        :type http_request: ~azure.purview.catalog.core.rest.HttpRequest
+        :keyword bool stream_response: Whether the response payload will be streamed. Defaults to False.
+        :return: The response of your network call. Does not do error handling on your response.
+        :rtype: ~azure.purview.catalog.core.rest.HttpResponse
+        """
+        request_copy = deepcopy(http_request)
+        path_format_arguments = {
+            'Endpoint': self._serialize.url("self._config.endpoint", self._config.endpoint, 'str', skip_quote=True),
+        }
+        request_copy.url = self._client.format_url(request_copy.url, **path_format_arguments)
+        if kwargs.pop("stream_response", False):
+            return _StreamContextManager(
+                client=self._client._pipeline,
+                request=request_copy,
+            )
+        pipeline_response = self._client._pipeline.run(request_copy._internal_request, **kwargs)
+        response = HttpResponse(
+            status_code=pipeline_response.http_response.status_code,
+            request=request_copy,
+            _internal_response=pipeline_response.http_response
+        )
+        response.read()
+        return response
+
+    def close(self):
+        # type: () -> None
+        self._client.close()
+
+    def __enter__(self):
+        # type: () -> PurviewCatalogClient
+        self._client.__enter__()
+        return self
+
+    def __exit__(self, *exc_details):
+        # type: (Any) -> None
+        self._client.__exit__(*exc_details)

sdk/purview/azure-purview-catalog/azure/purview/catalog/_configuration.py

@@ -0,0 +1,70 @@
+# coding=utf-8
+# --------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License. See License.txt in the project root for license information.
+# Code generated by Microsoft (R) AutoRest Code Generator.
+# Changes may cause incorrect behavior and will be lost if the code is regenerated.
+# --------------------------------------------------------------------------
+
+from typing import TYPE_CHECKING
+
+from azure.core.configuration import Configuration
+from azure.core.pipeline import policies
+
+from ._version import VERSION
+
+if TYPE_CHECKING:
+    # pylint: disable=unused-import,ungrouped-imports
+    from typing import Any
+
+    from azure.core.credentials import TokenCredential
+
+
+class PurviewCatalogClientConfiguration(Configuration):
+    """Configuration for PurviewCatalogClient.
+
+    Note that all parameters used to create this instance are saved as instance
+    attributes.
+
+    :param credential: Credential needed for the client to connect to Azure.
+    :type credential: ~azure.core.credentials.TokenCredential
+    :param endpoint: The catalog endpoint of your Purview account. Example: https://{accountName}.catalog.purview.azure.com.
+    :type endpoint: str
+    """
+
+    def __init__(
+        self,
+        credential,  # type: "TokenCredential"
+        endpoint,  # type: str
+        **kwargs  # type: Any
+    ):
+        # type: (...) -> None
+        if credential is None:
+            raise ValueError("Parameter 'credential' must not be None.")
+        if endpoint is None:
+            raise ValueError("Parameter 'endpoint' must not be None.")
+        super(PurviewCatalogClientConfiguration, self).__init__(**kwargs)
+
+        self.credential = credential
+        self.endpoint = endpoint
+        self.api_version = "2021-05-01-preview"
+        self.credential_scopes = kwargs.pop('credential_scopes', ['https://purview.azure.net/.default'])
+        kwargs.setdefault('sdk_moniker', 'purview-catalog/{}'.format(VERSION))
+        self._configure(**kwargs)
+
+    def _configure(
+        self,
+        **kwargs  # type: Any
+    ):
+        # type: (...) -> None
+        self.user_agent_policy = kwargs.get('user_agent_policy') or policies.UserAgentPolicy(**kwargs)
+        self.headers_policy = kwargs.get('headers_policy') or policies.HeadersPolicy(**kwargs)
+        self.proxy_policy = kwargs.get('proxy_policy') or policies.ProxyPolicy(**kwargs)
+        self.logging_policy = kwargs.get('logging_policy') or policies.NetworkTraceLoggingPolicy(**kwargs)
+        self.http_logging_policy = kwargs.get('http_logging_policy') or policies.HttpLoggingPolicy(**kwargs)
+        self.retry_policy = kwargs.get('retry_policy') or policies.RetryPolicy(**kwargs)
+        self.custom_hook_policy = kwargs.get('custom_hook_policy') or policies.CustomHookPolicy(**kwargs)
+        self.redirect_policy = kwargs.get('redirect_policy') or policies.RedirectPolicy(**kwargs)
+        self.authentication_policy = kwargs.get('authentication_policy')
+        if self.credential and not self.authentication_policy:
+            self.authentication_policy = policies.BearerTokenCredentialPolicy(self.credential, *self.credential_scopes, **kwargs)

sdk/purview/azure-purview-catalog/azure/purview/catalog/_version.py

@@ -0,0 +1,9 @@
+# coding=utf-8
+# --------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License. See License.txt in the project root for license information.
+# Code generated by Microsoft (R) AutoRest Code Generator.
+# Changes may cause incorrect behavior and will be lost if the code is regenerated.
+# --------------------------------------------------------------------------
+
+VERSION = "1.0.0b1"