Optimise iteration API

[Pimm]

Basic

This pull requests attempts to optimise the iterate method.

In this scenario, exactly 130 values are requested from the list payments endpoint, as the implementation correctly guesses that any additional values would be "wasted":

for await (let payment of mollieClient.payments.iterate().drop(30).take(100)) {
  console.log(payment);
}

In this scenario, a single request does not satisfy the demand. The Mollie API does not return pages of over 250 values. Therefore, the implementation requests two pages with 150 values each, rather than requesting 250 values twice "wasting" 200 values in the process:

for await (let payment of mollieClient.payments.iterate().take(300)) {
  console.log(payment);
}

Advanced

The iterate methods now return a lazy iterator: a container which creates and holds an upstream iterator when it is needed. As such, the implementation can watch for calls to take, drop, and filter and use that information to make an educated guess about the number of values which are to be consumed when the upstream iterator is created.

A demand of infinity is initially guessed. The chain is then traversed from right to left:

• take reduces the guess to the lowest value out of the current guess and the limit passed to take.
• drop increases the guess by the limit passed to drop.
• filter resets the guess to infinity.

Limitations

Limits applied through other means than take cannot be detected. The implementation does not optimise the following snippet:

const payments = [];
for await (let payment of mollieClient.payments.iterate()) {
  payments.push(payment);
  if (payments.length == 10) {
    break;
  }
}

Because there is no way to predict how many values will satisfy a certain filter, filter before take is not optimised:

// (There is no way to predict how many payments are required to end up with 10.)
for await (let payment of mollieClient.payments.iterate().filter(payment => Math.random() > .5).take(10)) {
  console.log(payment);
}

filter after take is OK, of course.

Because any number of new iterators can be created from any iterator (through drop, filter, map, and take), those iterators together form a (rooted) tree. The educated guess is derived from the path between the root and the leaf which triggers the creation of the upstream iterator.

This does mean that in the following example, this class incorrectly guesses that only 10 values are required when in actuality 12 are required:

const iterator = mollieClient.payments.iterate();
for await (let payment of iterator.take(10)) {
  console.log(payment);
}
// (The endpoint has already been called, therefore this take is ignored.)
for await (let payment of iterator.take(2)) {
  console.log(payment);
}

[dsempel]

Clever optimization!