Qyu

An in-memory queue in JavaScript

This queue can run asynchronous jobs sequentially (with rateLimit = null) or concurrently (with maximum number of jobs that can be run per second, in rateLimit).

Documentation: qyu API docs (Generated by JSDoc)

⚠️ This implementation is not intended for production use, and the underlying data structure is not safe to use in a multi-threaded environment.

Prerequisites

• Node.js ≧ 7.6 (with support of async/await)

Install and testing instructions

$ npm install adrienjoly/algolia-qyu
$ cd node_modules/qyu
$ npm install
$ npm test

Design

Example of use

Here's a minimal example of use:

const qyu = require('qyu');

const q = qyu({
  rateLimit: 50, // maximum number of jobs being processed by second
  statsInterval: 300 // When stat event is sent, in ms
});

q.on('done', ({jobId, jobResult, res}) => {
  console.log(`Job done ${jobId}`); // `jobId` is generated by `qyu`
});

q.on('error', ({jobId, error}) => {
  console.log(`Job ${jobId} threw an error: ${error.message}`);
});

q.on('drain', () => {
  console.log('No more jobs to do');
});

q.on('stats', ({nbJobsPerSecond}) => {
  console.log(`${nbJobsPerSecond} jobs/s processed`)
});

q.push(job, { // job is a function returning a promise to indicate when the job is done
  priority: 1, // from 1 to 10, 1 being the highest priority
}); // returns a promise (which resolves with {jobId, jobResult})

q.pause(); // returns a promise resolved when `q` has paused (no jobs being processed)
q.start(); // returns a promise resolved when `q` has started (first time) or unpaused

// example job:
async function job() {
  await wait(30);
  return {Hello: 'world!'} // That's the `jobResult`
}

function wait(ms) {
  return new Promise(resolve) {
    setTimeout(resolve, ms)
  }
}