data.md

I've had to answer flow control issues repeatedly. Both in my own code-base and as answers to StackOverflow questions. All these problems can be handled with one of two methods. One issue is dealing with nested asynchronous tasks and the answer to this is to use middleware. The other issue is aggregating many asynchronous tasks into one end point and the answer is this is to use function execution delay.

I've build a tiny utility function called after to handle aggregating of asynchronous tasks. I'll walk you through how to use it in this post. I will get around to building a generic middleware component that can be used to chain asynchronous tasks together effectively, it will be inspired by connect.

Description

Let's start with a description of after. Below is an illustrated example of using it

// Create a single end point to aggregate two asynchronous tasks into.
var cb = after(2, function _allFinished(file1, file2) {
  console.log("all finished");
});

// run two asynchronous tasks and pass the callback to them
fs.readFile("file1.js", cb);
fs.readFile("file2.js", cb);

var f = after(count, callback)

By calling after with a count and a callback you create a new function f that will call the callback after it's been called count times. That is all.

The callback will receive the data passed to f from each call as separate parameters. If you pass one argument to f, then callback receives that argument, if you pass multiple arguments to f then callback will receive an array of those arguments.

after was inspired by _.after

Source

module.exports = function _after(count, f) {
  var c = 0, results = [];
  return function _callback() {
    switch (arguments.length) {
      case 0: results.push(null); break;
      case 1: results.push(arguments[0]); break;
      default: results.push(Array.prototype.slice.call(arguments)); break;
    }
    if (++c === count) {
      f.apply(this, results);
    }
  };
};

Usecases

Let's say we have a function that needs to call asynchronous actions on arbitrary input and then continue when they all finish.

var loadFolder = function(folder, cb) {
  // read all files in folder
  readFolder(folder, function(files) {
    // set the cb to fire after all files have loaded
    var _cb = after(files.length, cb);

    // load and run each file
    files.forEach(function(file) {
      require(file).on("loaded", _cb);
    });
  });
};

Based on a real example

As you can see, it's very useful to aggregate the result of many actions into one callback.

Let's take a different example

You want to make two database queries. and then render some data.

app.get(route, function(req, res) {
  var handleDatabase = after(2, function (res1, res2) {
    res.render('home.ejs', { locals: { r1: res1, r2: res2 }):
  });

  db.execute(sql1).on('result', handleDatabase);
  db.execute(sql2).on('result', handleDatabase);
});

Here we are simply cleaning up the code. If you want to do asynchronous queries for data then you have to do some reference counting to make sure you don't render anything before they both finish. after does the reference counting for you and in one place. It's a tidy way to run a block of code when you know you are ready.

This is the first half of what people use flow control for. The other need for flow control is to stop endless nesting of callbacks where you do 7/8 asynchronous tasks in a row and end up being nested 7/8 tasks deep.

One proposed solution would be to do something like flow does

Flow(function() {
  doSomethingAsync(this.cb);
}, function() {
  doSomethignAsync(this.cb);
}, ... , function() {
  finish();
});

This basically made the nesting disappear and placed all the asynchronous calls as a neat queue. I propose an alternative which is to use middleware. We already effectively use middleware for routing (using connect).

Below is an example from my blog code

m = [
  middle.requireLogin,
  middle.beRaynos,
  middle.validate,
  middle.checkId,
  middle.validatePost,
  middle.getPostById,
  middle.checkPostExistance,
  middle.savePost
];
app.put("/blog/:postId", m, middle.redirectToPost);

Here we are doing 9 actions in order using middleware syntax where we have access to a next variable. Feel free to read the route middleware section on the express guide to understand middleware. I will talk about middleware in detail as part of my blog post series.

Eventually I will write a middleware library and there will be another blog post when I do.