Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BigTable asynchronous row handling #1715

Closed
arbesfeld opened this issue Oct 18, 2016 · 2 comments
Closed

BigTable asynchronous row handling #1715

arbesfeld opened this issue Oct 18, 2016 · 2 comments
Assignees
Labels
api: bigtable Issues related to the Bigtable API. type: question Request for information or clarification. Not an issue.

Comments

@arbesfeld
Copy link
Contributor

arbesfeld commented Oct 18, 2016

Is there a way to asynchronously handle rows with a BigTable range query? What I'm trying to do is something like this:

table.getRows().on('data', async row => {
  await doSomeExpensiveWork(row);
});
@callmehiphop callmehiphop added type: question Request for information or clarification. Not an issue. api: bigtable Issues related to the Bigtable API. labels Oct 18, 2016
@callmehiphop
Copy link
Contributor

callmehiphop commented Oct 18, 2016

What about something like this?

$ npm install --save through2
var through = require('through2');

table.createReadStream()
  .on('error', console.error)
  .pipe(through.obj((row, _, next) => {
    doSomeExpensiveWork()
      .then(() => next(null), next);
  })
  .on('finish', function() {
    // all done!
  });

@stephenplusplus
Copy link
Contributor

Yep, that's what I would say. It will not pull more data from the stream until you're done processing the last result.

In case users of other APIs see this, this will work for any object stream from our APIs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigtable Issues related to the Bigtable API. type: question Request for information or clarification. Not an issue.
Projects
None yet
Development

No branches or pull requests

3 participants