Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PubSub topic.create method should raise a Conflict error instead of RetryError/GaxError when topic exists #3175

Closed
lucmult opened this issue Mar 21, 2017 · 8 comments · Fixed by #3443
Assignees
Labels
api: core api: pubsub Issues related to the Pub/Sub API. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release.

Comments

@lucmult
Copy link
Contributor

lucmult commented Mar 21, 2017

Following the documentation from PubSub site if you run the example 2x it will generate the error:

GaxError(Exception occurred in retry method that was not classified as transient, caused by <_Rendezvous of RPC that terminated with (StatusCode.ALREADY_EXISTS, Resource already exists in the project (resource=my-new-topic).)>) <class 'google.gax.errors.RetryError'>

However it should raise Conflict Error (google.cloud.exceptions.Conflict), because the API returns a 409 response.
You can see the status code using API Explorer:
https://developers.google.com/apis-explorer/#search/pubsub/pubsub/v1/pubsub.projects.topics.create

409  
{
 "error": {
  "code": 409,
  "message": "Resource already exists in the project (resource=my-new-topic).",
  "status": "ALREADY_EXISTS",
...
  }
}

Full traceback on Python:

>>> topic.create()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/google/home/lucmult/.virtualenvs/pubsub/local/lib/python2.7/site-packages/google/cloud/pubsub/topic.py", line 155, in create
    api.topic_create(topic_path=self.full_name)
  File "/usr/local/google/home/lucmult/.virtualenvs/pubsub/local/lib/python2.7/site-packages/google/cloud/pubsub/_gax.py", line 104, in topic_create
    topic_pb = self._gax_api.create_topic(topic_path)
  File "/usr/local/google/home/lucmult/.virtualenvs/pubsub/local/lib/python2.7/site-packages/google/cloud/gapic/pubsub/v1/publisher_client.py", line 278, in create_topic
    return self._create_topic(request, options)
  File "/usr/local/google/home/lucmult/.virtualenvs/pubsub/local/lib/python2.7/site-packages/google/gax/api_callable.py", line 419, in inner
    return api_caller(api_call, this_settings, request)
  File "/usr/local/google/home/lucmult/.virtualenvs/pubsub/local/lib/python2.7/site-packages/google/gax/api_callable.py", line 407, in base_caller
    return api_call(*args)
  File "/usr/local/google/home/lucmult/.virtualenvs/pubsub/local/lib/python2.7/site-packages/google/gax/api_callable.py", line 368, in inner
    return a_func(*args, **kwargs)
  File "/usr/local/google/home/lucmult/.virtualenvs/pubsub/local/lib/python2.7/site-packages/google/gax/retry.py", line 126, in inner
    ' classified as transient', exception)
google.gax.errors.RetryError: GaxError(Exception occurred in retry method that was not classified as transient, caused by <_Rendezvous of RPC that terminated with (StatusCode.ALREADY_EXISTS, Resource already exists in the project (resource=my-new-topic).)>)
@dhermes
Copy link
Contributor

dhermes commented Mar 21, 2017

@lukesneeringer This is part of a more general question: how / where should we re-wrap GaxError?

@lucmult
Copy link
Contributor Author

lucmult commented Mar 21, 2017

For anybody affected by this, to handle ALREADY_EXISTS exception you can use the following scheme:

try:
  # create topic
except Conflict as error:
   # handle the current conflict exception
except RetryException as error:
   status_code = error.cause.code()
   if status_code == status_code.ALREADY_EXISTS:
      # handle the already exists
   else:
      # nothing that we want just re-raise the exception
      raise

@lukesneeringer
Copy link
Contributor

@lukesneeringer This is part of a more general question: how / where should we re-wrap GaxError?

The correct answer is to have GAX raise these different kinds of errors, rather than the catch-all GaxError. It could import them from google.cloud.exceptions.

@lukesneeringer lukesneeringer self-assigned this Mar 21, 2017
@lukesneeringer lukesneeringer added api: core priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. status: acknowledged labels Mar 21, 2017
@dhermes
Copy link
Contributor

dhermes commented Mar 21, 2017

@lukesneeringer That would be much preferred to the current ad-hoc approach (only in use in one subpackage, though some other remappings also exist).

@marcboissonneaultjda
Copy link

@dhermes While your team is fixing the issue, can you to fix the subscription_create issue at the same time?
It's exactly the same issue but for subscription and I completely forgot to mention it in my support case with @lucmult

@tseaver
Copy link
Contributor

tseaver commented May 19, 2017

@lukesneeringer Generalizing the solution from datastore requires moving datastore._gax._catch_remap_gax_error into core, most obviously core.exceptions. Doing that requires adding a from google.cloud.core to google.gax. We have unconditional dependencies to google.gax in other packages: would that be a problem for core?

@tseaver tseaver added the api: pubsub Issues related to the Pub/Sub API. label May 19, 2017
@tseaver
Copy link
Contributor

tseaver commented May 19, 2017

The issue within pubsub seems to be that the back-end API shifted from returning FAILED_PRECONDITION to ALREADY_EXISTS: we have handling which raises Conflict for the older code.

We should add a system test (or extend and existing one) to ensure that we don't regress whatever fix we apply now.

tseaver added a commit that referenced this issue Jun 7, 2017
* Add testing support for 'ALREADY_EXISTS' gRPC error code.

* Cover both possible gRPC conflict error codes.

Closes #3175.

* Exercise conflict-on-create in systests for topic/sub/snap.
landrito pushed a commit to landrito/google-cloud-python that referenced this issue Aug 21, 2017
* Add testing support for 'ALREADY_EXISTS' gRPC error code.

* Cover both possible gRPC conflict error codes.

Closes googleapis#3175.

* Exercise conflict-on-create in systests for topic/sub/snap.
landrito pushed a commit to landrito/google-cloud-python that referenced this issue Aug 22, 2017
* Add testing support for 'ALREADY_EXISTS' gRPC error code.

* Cover both possible gRPC conflict error codes.

Closes googleapis#3175.

* Exercise conflict-on-create in systests for topic/sub/snap.
landrito pushed a commit to landrito/google-cloud-python that referenced this issue Aug 22, 2017
* Add testing support for 'ALREADY_EXISTS' gRPC error code.

* Cover both possible gRPC conflict error codes.

Closes googleapis#3175.

* Exercise conflict-on-create in systests for topic/sub/snap.
@Saurav-Jain16
Copy link

Saurav-Jain16 commented Jan 19, 2019

Error is coming in below code when starting the server "(node:6924) UnhandledPromiseRejectionWarning: Error: 6 ALREADY_EXISTS: Resource already exists in the project (resource=serverone)."

Any idea why this issue is coming. Same code is working on other server

'use strict';

const http = require('http');

// Imports the Google Cloud client library
const PubSub = require('@google-cloud/pubsub');

// Your Google Cloud Platform project ID
const projectId = 'projectId';

// Instantiates a client
const pubsub = PubSub({
projectId: projectId,
keyFilename:'keyFilename.json'
});

// Reference a topic that has been previously created.
var topic = pubsub.topic('dashboardConnector');

createFlowControlledSubscription('dashboardConnector','serverone',1,10000);

function createFlowControlledSubscription (topicName, subscriptionName, maxInProgress, maxBytes) {
// Instantiates a client
// const pubsub = PubSub();

// References an existing topic, e.g. "my-topic"
const topic = pubsub.topic(topicName);

// Creates a new subscription, e.g. "my-new-subscription"
// Note that flow control configurations are not persistent
return topic.createSubscription(subscriptionName, {
flowControl: {
maxBytes: maxBytes,
maxMessages: maxInProgress
}
})
.then((results) => {
const subscription = results[0];

  subscription.on('message', function(message) {
    var attrs=message.attributes;
    var data =JSON.parse(message.data.toString('utf8'));

     
	
       message.ack();
    });

  console.log(`Subscription ${subscription.name} created with a maximum of ${maxInProgress} unprocessed messages.`);
  return subscription;
});

}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: core api: pubsub Issues related to the Pub/Sub API. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants