Skip to content

Commit

Permalink
feat!: migrate to use microgen (#71)
Browse files Browse the repository at this point in the history
* feat!: migrate to use microgen

* update

* update

* update

* update
  • Loading branch information
arithmetic1728 authored Aug 10, 2020
1 parent f6eb330 commit ce25e94
Show file tree
Hide file tree
Showing 196 changed files with 48,837 additions and 47,388 deletions.
13 changes: 6 additions & 7 deletions packages/google-cloud-dataproc/.coveragerc
Original file line number Diff line number Diff line change
Expand Up @@ -21,15 +21,14 @@ branch = True
[report]
fail_under = 100
show_missing = True
omit = google/cloud/dataproc/__init__.py
exclude_lines =
# Re-enable the standard pragma
pragma: NO COVER
# Ignore debug-only repr
def __repr__
# Ignore abstract methods
raise NotImplementedError
omit =
*/gapic/*.py
*/proto/*.py
*/core/*.py
*/site-packages/*.py
# Ignore pkg_resources exceptions.
# This is added at the module level as a safeguard for if someone
# generates the code and tries to run it without pip installing. This
# makes it virtually impossible to test properly.
except pkg_resources.DistributionNotFound
10 changes: 6 additions & 4 deletions packages/google-cloud-dataproc/README.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Python Client for Google Cloud Dataproc API
===========================================

|ga| |pypi| |versions|
|ga| |pypi| |versions|

`Google Cloud Dataproc API`_: Manages Hadoop-based clusters and jobs on Google Cloud Platform.

Expand Down Expand Up @@ -49,11 +49,13 @@ dependencies.

Supported Python Versions
^^^^^^^^^^^^^^^^^^^^^^^^^
Python >= 3.5
Python >= 3.6

Deprecated Python Versions
^^^^^^^^^^^^^^^^^^^^^^^^^^
Python == 2.7. Python 2.7 support will be removed on January 1, 2020.
Python == 2.7.

The last version of this library compatible with Python 2.7 is google-cloud-dataproc==1.1.0.


Mac/Linux
Expand Down Expand Up @@ -107,4 +109,4 @@ Next Steps
- Read the `Client Library Documentation`_ for Google Cloud Dataproc API
API to see other available methods on the client.
- Read the `Product documentation`_ to learn more about the product and see
How-to Guides.
How-to Guides.
168 changes: 168 additions & 0 deletions packages/google-cloud-dataproc/UPGRADING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
# 2.0.0 Migration Guide

The 2.0 release of the `google-cloud-dataproc` client is a significant upgrade based on a [next-gen code generator](https://github.com/googleapis/gapic-generator-python), and includes substantial interface changes. Existing code written for earlier versions of this library will likely require updates to use this version. This document describes the changes that have been made, and what you need to do to update your usage.

If you experience issues or have questions, please file an [issue](https://github.com/googleapis/python-dataproc/issues).

## Supported Python Versions

> **WARNING**: Breaking change
The 2.0.0 release requires Python 3.6+.


## Method Calls

> **WARNING**: Breaking change
Methods expect request objects. We provide a script that will convert most common use cases.

* Install the library

```py
python3 -m pip install google-cloud-dataproc
```

* The script `fixup_dataproc_v1_keywords.py` is shipped with the library. It expects an input directory (with the code to convert) and an empty destination directory.

```sh
$ fixup_dataproc_v1_keywords.py --input-directory .samples/ --output-directory samples/
```

**Before:**
```py
from google.cloud import dataproc

client = dataproc.ClusterControllerClient()

clusters = client.list_clusters(project_id="project_id", region="region")
```


**After:**
```py
from google.cloud import dataproc

client = dataproc.ClusterControllerClient()

clusters = client.list_clusters(request={
'project_id' : "project_id", 'region' : "region"
})
```

### More Details

In `google-cloud-dataproc<2.0.0`, parameters required by the API were positional parameters and optional parameters were keyword parameters.

**Before:**
```py
def get_cluster(
self,
project_id,
region,
cluster_name,
retry=google.api_core.gapic_v1.method.DEFAULT,
timeout=google.api_core.gapic_v1.method.DEFAULT,
metadata=None,
):
```

In the 2.0.0 release, all methods have a single positional parameter `request`. Method docstrings indicate whether a parameter is required or optional.

Some methods have additional keyword only parameters. The available parameters depend on the [`google.api.method_signature` annotation](https://github.com/googleapis/googleapis/blob/master/google/cloud/dataproc/v1/clusters.proto#L88) specified by the API producer.


**After:**
```py
def get_cluster(
self,
request: clusters.GetClusterRequest = None,
*,
project_id: str = None,
region: str = None,
cluster_name: str = None,
retry: retries.Retry = gapic_v1.method.DEFAULT,
timeout: float = None,
metadata: Sequence[Tuple[str, str]] = (),
) -> clusters.Cluster:
```

> **NOTE:** The `request` parameter and flattened keyword parameters for the API are mutually exclusive.
> Passing both will result in an error.

Both of these calls are valid:

```py
response = client.get_cluster(
request={
"project_id": project_id,
"region": region,
"cluster_name": cluster_name
}
)
```

```py
response = client.get_cluster(
project_id=project_id,
region=region,
cluster_name=cluster_name
)
```

This call is invalid because it mixes `request` with a keyword argument `cluster_name`. Executing this code
will result in an error.

```py
response = client.get_cluster(
request={
"project_id": project_id,
"region": region
},
cluster_name=cluster_name
)
```



## Enums and Types


> **WARNING**: Breaking change
The submodules `enums` and `types` have been removed.

**Before:**
```py

from google.cloud import dataproc

status = dataproc.enums.ClusterStatus.State.CREATING
cluster = dataproc.types.Cluster(cluster_name="name")
```


**After:**
```py
from google.cloud import dataproc

status = dataproc.ClusterStatus.State.CREATING
cluster = dataproc.Cluster(cluster_name="name")
```

## Path Helper Methods
The following path helper methods have been removed. Please construct the paths manually.

```py
project = 'my-project'
location = 'project-location'
region = 'project-region'
workflow_template = 'template'
autoscaling_policy = 'policy'

location_path = f'projects/{project}/locations/{location}'
region_path = f'projects/{project}/regions/{region}'
workflow_template_path = f'projects/{project}/regions/{region}/workflowTemplates/{workflow_template}'
autoscaling_policy_path = f'projects/{project}/locations/{location}/autoscalingPolicies/{autoscaling_policy}'
```
1 change: 1 addition & 0 deletions packages/google-cloud-dataproc/docs/UPGRADING.md
15 changes: 15 additions & 0 deletions packages/google-cloud-dataproc/docs/dataproc_v1/services.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
Services for Google Cloud Dataproc v1 API
=========================================

.. automodule:: google.cloud.dataproc_v1.services.autoscaling_policy_service
:members:
:inherited-members:
.. automodule:: google.cloud.dataproc_v1.services.cluster_controller
:members:
:inherited-members:
.. automodule:: google.cloud.dataproc_v1.services.job_controller
:members:
:inherited-members:
.. automodule:: google.cloud.dataproc_v1.services.workflow_template_service
:members:
:inherited-members:
5 changes: 5 additions & 0 deletions packages/google-cloud-dataproc/docs/dataproc_v1/types.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Types for Google Cloud Dataproc v1 API
======================================

.. automodule:: google.cloud.dataproc_v1.types
:members:
15 changes: 15 additions & 0 deletions packages/google-cloud-dataproc/docs/dataproc_v1beta2/services.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
Services for Google Cloud Dataproc v1beta2 API
==============================================

.. automodule:: google.cloud.dataproc_v1beta2.services.autoscaling_policy_service
:members:
:inherited-members:
.. automodule:: google.cloud.dataproc_v1beta2.services.cluster_controller
:members:
:inherited-members:
.. automodule:: google.cloud.dataproc_v1beta2.services.job_controller
:members:
:inherited-members:
.. automodule:: google.cloud.dataproc_v1beta2.services.workflow_template_service
:members:
:inherited-members:
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Types for Google Cloud Dataproc v1beta2 API
===========================================

.. automodule:: google.cloud.dataproc_v1beta2.types
:members:
6 changes: 0 additions & 6 deletions packages/google-cloud-dataproc/docs/gapic/v1/api.rst

This file was deleted.

5 changes: 0 additions & 5 deletions packages/google-cloud-dataproc/docs/gapic/v1/types.rst

This file was deleted.

6 changes: 0 additions & 6 deletions packages/google-cloud-dataproc/docs/gapic/v1beta2/api.rst

This file was deleted.

5 changes: 0 additions & 5 deletions packages/google-cloud-dataproc/docs/gapic/v1beta2/types.rst

This file was deleted.

19 changes: 14 additions & 5 deletions packages/google-cloud-dataproc/docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,20 @@ API Reference
.. toctree::
:maxdepth: 2

gapic/v1/api
gapic/v1/types
gapic/v1beta2/api
gapic/v1beta2/types
dataproc_v1/services
dataproc_v1/types
dataproc_v1beta2/services
dataproc_v1beta2/types

Migration Guide
---------------

See the guide below for instructions on migrating to the 2.x release of this library.

.. toctree::
:maxdepth: 2

UPGRADING

Changelog
---------
Expand All @@ -21,4 +30,4 @@ For a list of all ``google-cloud-dataproc`` releases:
.. toctree::
:maxdepth: 2

changelog
changelog
Loading

0 comments on commit ce25e94

Please sign in to comment.