Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JS Driver: promise friendly cursor.eachAsync() #4784

Merged
merged 4 commits into from Sep 21, 2015

Conversation

marshall007
Copy link
Contributor

Fixes #4780. Unfortunately, there was no way to implement this as an overload to the existing cursor.each() since there's no way to disambiguate the argument style.

Here's some example usage:

cursor.eachAsync(function (row) {
  if (/* some condition */) {
    // if you return a promise, it will be awaited
    // before the cursor continues iteration
    return asyncRowHandler(row.id);
  }
  else {
    // iteration can be stopped early by returning a
    // promise that is later rejected or simply by throw'ing
    return Promise.reject('Stop iteration early.')
  }
})
.then(function () {
  // `cursor.eachAsync()` returns a promise that is resolved
  // once all rows have been read from the cursor
});

@jorrit
Copy link

jorrit commented Sep 2, 2015

If I read the coffeescript correctly this method will sequentialize all iterations by invoking the next callback when the previous one is resolved. Isn't it possible to execute all iterations in parallel with a Promise.all-like approach?

@marshall007
Copy link
Contributor Author

@jorrit that would be easier to do now, at least:

var results = [];

cursor.eachAsync(function (row) {
  results.push(asyncRowHandler(row.id));
  // as long as we don't return the promise
  // the next row will be read immediately
})
.then(function () {
  return Promise.all(results).then(function (data) { /* ... */ });
});

@jorrit
Copy link

jorrit commented Sep 2, 2015

Personally I see no benefit in eachAsync supporting a callback that returns Promises when it serializes these promises. Maybe make it configurable with an options parameter?

@coffeemug
Copy link
Contributor

/cc @deontologician @Tryneus

@deontologician
Copy link
Contributor

はい

@larkost
Copy link
Collaborator

larkost commented Sep 2, 2015

I would make me (the testing guy) particularly happy if you added appropriate tests to test/rql_test/connections/cursor.mocha

return p
else
throw new err.ReqlDriverError "First argument to `next` must be a function or undefined."
return Promise.fromNode(fn).nodeify(cb)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Damn, that is a nice cleanup. I need to do that in the other places we do this manually...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@deontologician thanks! In my last commit I went ahead and refactored cursor.toArray into a one-liner as well.

@marshall007
Copy link
Contributor Author

@larkost I already copied the cursor.each tests and made them work for cursor.eachAsync, but I'll definitely add a couple more since those are fairly sparse.

@deontologician
Copy link
Contributor

Maybe make it configurable with an options parameter?

I like this interface. It seems to make the common case easy. When I think of iteration with each, I think of them executing in order. Given that you can just collect them and run them asynchronously if you want, it seems ok not to provide an option.

It looks good to me overall. I've played with the code a bit and it works like I'd expect. Since this is an api change, I'd like to let it breath for a bit before merging it. Maybe get some opinions from other js guys that hang around here like @neumino and @joaojeronimo ...

@marshall007
Copy link
Contributor Author

@larkost just added tests covering all the cases outlined in my opening comment and added a "stop iteration early" test to cursor.each. Let me know if that doesn't suffice.

@deontologician somewhat unrelated to this PR, the original cursor.each implementation appears to read an extra row off the cursor after iteration has already been stopped. I added a fix for that and the tests still pass, but I wanted to make sure I wasn't overlooking something intentional.

@deontologician
Copy link
Contributor

Looks like it was part of removing hasNext() in commit 190c8c2 . It's possible reading one more is for ensuring the cursor has retrieved everything there is off the server, but I haven't looked too closely yet at the code.

@marshall007
Copy link
Contributor Author

Ok, that makes me pretty certain it was a bug. We weren't even checking the result of that last (unintentional) iteration, so I don't see how it could impact anything.

@neumino
Copy link
Member

neumino commented Sep 3, 2015

@deontologician -- Adding a new method seems reasonable to me and I can see how useful it can be.

Maybe there's a better name than eachAsync since people may thing that each is synchronous, but I have to admit that I can't come up with a better name.
Bluebird's doc calls map(... {concurrency: 1}) a static map, but it doesn't sound good :)

@deontologician
Copy link
Contributor

Ok, I'm ready to merge this, but I was wondering what people thought of the name "eachPromise" for the name. I think I agree with @neumino that since both each implementations are async, it might be misleading to call it eachAsync. @marshall007 thoughts? (or a better name)

@marshall007
Copy link
Contributor Author

@deontologician I think at the next opportunity to introduce breaking changes (i.e. 2.2) we should consider changing the current signature of cursor.each so that it can once again support both styles:

cursor.each(function(err, row), function()) // current

cursor.each(function(row, cb), function(err)) // new callback-style
cursor.each(function(row)) // new promise-style

I wouldn't mind waiting to merge all this at once under cursor.each. If you want to go ahead and get it released, I think the -Async suffix is fairly standard and I definitely prefer that over eachPromise.

https://github.com/petkaantonov/bluebird/blob/master/API.md#promisification

Notice examples like fs.readFileAsync in their docs even though fs.readFile is asynchronous as well.

@deontologician
Copy link
Contributor

Hmm. Alright, in that case I think we should just merge this as-is then. Breaking backwards compatibility in something this core is something I would expect in a major release (like 3.0). I think adding the extra method is a good compromise in the meantime.

deontologician added a commit that referenced this pull request Sep 21, 2015
JS Driver: promise friendly `cursor.eachAsync()`
@deontologician deontologician merged commit 7a0ebd5 into rethinkdb:next Sep 21, 2015
larkost added a commit that referenced this pull request Sep 23, 2015
branch: larkost/4787-yaml-parser
review: CR 3237
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

(Javascript) Make each() return a Promise
7 participants