Support an action that takes an asynchronous callback

[shreevatsa]

I don't know enough about JavaScript to know whether this is the same issue as #1 (though I suspect it is), but let me describe it anyway :-)

All the examples end with pandoc.stdio(action) where action is a function like:

function action(type, value, format, meta) {
    // ...
    return something;
}

In my case, inside action I need to call a function that takes a callback (as is typical for Node):

function action(type, value, format, meta) {
    // ...
   doAsync(value, function () { ... });
}

So I won't have something to return from inside action.

I am right now trying to hack it out by copying and editing much of the internals of pandoc-filter.
(Specifically, stdio which is an alias for toJSONFilter calls filter which calls walk and it appears I need to hack a copy of that function: https://github.com/mvhenderson/pandoc-filter-node/blob/be9d69ac/index.js#L47,L79 .)

But I imagine this is a common use case, and it would be better if pandoc-filter supported this.

[shreevatsa]

Here's a hacked version of the index.js code, that lets you use an action that returns a promise (or is an async function). It requires Node 7.6 or later (as it uses async/await):

async function walk(x, action, format, meta) {
  if (Array.isArray(x)) {
    const array = [];
    for (const item of x) {
      if (item === Object(item) && item.t) {
        var res = await action(item.t, item.c || [], format, meta);
        if (!res) {
          array.push(await walk(item, action, format, meta));
        }
        else if (Array.isArray(res)) {
          for (const z of res) {
            array.push(await walk(z, action, format, meta));
          };
        }
        else {
          array.push(await walk(res, action, format, meta));
        }
      }
      else {
        array.push(await walk(item, action, format, meta));
      }
    }
    return array;
  }
  else if (x === Object(x)) {
    var obj = {};
    for (const k of Object.keys(x)) {
      obj[k] = await walk(x[k], action, format, meta);
    }
    return obj;
  }
  return x;
}

async function filter(data, action, format) {
  return await walk(data, action, format, data.meta || data[0].unMeta);
}

function toJSONFilter(action) {
  require('get-stdin')(function (json) {
    var data = JSON.parse(json);
    var format = (process.argv.length > 2 ? process.argv[2] : '');
    filter(data, action, format).then(output => process.stdout.write(JSON.stringify(output)));
  });
}

So when pandoc.stdio(fooAction) doesn't work because fooAction returns a promise, you can instead use the above toJSONFilter(fooAction).

To repeat, I just hacked up whatever works, and I don't understand enough of the implications here to say whether this is a good idea. (For example, may it be better for performance to fire off calls to the action in parallel, or at least more in parallel than the above?) But it does the job, for me.

[mvhenderson]

@shreevatsa Thank you for this! I'll get it folded into pandoc-filter over the next few weeks, along with plan Promise and traditional callback support for those using older versions of node.

[ds82]

When can you merge this? @mvhenderson

[mvhenderson]

Thanks for the reminder - I'll get to it in the next few days.

…

Sent from my iPhone

On Jul 2, 2018, at 7:48 AM, Dennis ***@***.***> wrote: When can you merge this? @mvhenderson — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[mvhenderson]

@ds82 - Folded in code from @shreevatsa as *Async functions and published to NPM as v1.0.0. See README and the test/spec/async for example. Let me know if it works for you. Requires node with async/await support.