Skip to content

Conversation

@stephenplusplus
Copy link
Contributor

Fixes #379
Fixes #377

Topics have a new topic.publishBatch method that takes an array of messages in any format (string, object, or raw [like what would be sent to topic.publishRaw).

Subscriptions accept a new parameter, maxCount, to subscription.pull which triggers a batched pull for messages, up to the count specified.

@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Feb 9, 2015
@stephenplusplus stephenplusplus changed the title Spp pubsub batch methods Implement publishBatch and pullBatch Feb 9, 2015
@stephenplusplus stephenplusplus added the api: pubsub Issues related to the Pub/Sub API. label Feb 9, 2015

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

@stephenplusplus
Copy link
Contributor Author

Please give another look.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

@ryanseys
Copy link
Contributor

Overall looks great! Just a couple concerns this won't really fix the race-y tests we've been having. How's your experience running them on your machine?

Also, seems unnecessary to rename maxEvents to maxCount... maybe if we are going to rename, just rename it to max ? Just a thought. I wasn't sure the motivation for that change.

@ryanseys
Copy link
Contributor

Yeah I ran the regression tests and got the following error:

1) pubsub Subscription should receive the published message:
     Uncaught Error: The subscription has no more messages available (subscription="/subscriptions/project-id/sub1").
      at handleResp (/Users/ryanseys/gcloud-node/lib/common/util.js:176:14)
      at Request._callback (/Users/ryanseys/gcloud-node/lib/common/util.js:406:11)
      at Request.self.callback (/Users/ryanseys/gcloud-node/node_modules/request/request.js:344:22)
      at Request.emit (events.js:110:17)
      at Request.<anonymous> (/Users/ryanseys/gcloud-node/node_modules/request/request.js:1239:14)
      at Request.emit (events.js:129:20)
      at IncomingMessage.<anonymous> (/Users/ryanseys/gcloud-node/node_modules/request/request.js:1187:12)
      at IncomingMessage.emit (events.js:129:20)
      at _stream_readable.js:908:16
      at process._tickCallback (node.js:355:11)

@ryanseys
Copy link
Contributor

I just put the pull in the callback and it worked great!

it('should receive the published message', function(done) {

      var subscription = topic.subscription(subscriptions[0].name);

      topic.publish([
        { data: new Buffer('hello').toString('base64') },
        { data: new Buffer('hello').toString('base64') },
        { data: new Buffer('hello').toString('base64') },
        { data: new Buffer('hello').toString('base64') },
        { data: new Buffer('hello').toString('base64') },
        { data: new Buffer('hello').toString('base64') },
        { data: new Buffer('hello').toString('base64') },
        { data: new Buffer('hello').toString('base64') },
        { data: new Buffer('hello').toString('base64') },
        { data: new Buffer('hello').toString('base64') }
      ], function() {
        subscription.pull({
          returnImmediately: true,
          maxCount: 1
        }, function(err, msg) {
          assert.ifError(err);
          assert.equal(msg.data, 'hello');
          subscription.ack(msg.ackId, done);
        });
      });
    });

@stephenplusplus
Copy link
Contributor Author

Also, seems unnecessary to rename maxEvents to maxCount... maybe if we are going to rename, just rename it to max ? Just a thought. I wasn't sure the motivation for that change.

As a user of our Pub/Sub abstraction API, I'm not thinking "I need to pull for events", I'm thinking "I need to pull for messages". Our users will likely never interface directly with the upstream API, so I don't place a lot of value on being consistent with it in places where I think we can make it more logical. RE: max vs maxCount, I'd rather use maxCount for maximum clarity. Other places in our library use maxResults for example. (Maybe maxResults is better than maxCount?)

@stephenplusplus stephenplusplus force-pushed the spp--pubsub-batch-methods branch from 336d210 to 02d3f85 Compare February 16, 2015 13:42
@stephenplusplus
Copy link
Contributor Author

Updated the PR to move pulls to the publish callbacks.

@ryanseys
Copy link
Contributor

Yeah I think consistency would be nice here. maxResults makes sense too.

@stephenplusplus stephenplusplus force-pushed the spp--pubsub-batch-methods branch from 02d3f85 to 0f84cdd Compare February 17, 2015 16:54
@ryanseys
Copy link
Contributor

It this ready for another look?

@stephenplusplus
Copy link
Contributor Author

Not yet.

@stephenplusplus stephenplusplus force-pushed the spp--pubsub-batch-methods branch from 367753e to 6898811 Compare February 19, 2015 18:46
@stephenplusplus
Copy link
Contributor Author

Okay, ready for a look :)

This comment was marked as spam.

This comment was marked as spam.

sofisl pushed a commit that referenced this pull request Jan 27, 2026
sofisl pushed a commit that referenced this pull request Jan 27, 2026
sofisl pushed a commit that referenced this pull request Jan 28, 2026
GautamSharda pushed a commit that referenced this pull request Feb 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: pubsub Issues related to the Pub/Sub API. cla: yes This human has signed the Contributor License Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement publishBatch and pullBatch methods Looks like the build is broken?

4 participants