Running Asynchronous JavaScript Code in Sequence with Async Waterfall - Part 2

Project Files

Click here to download the project files.
Make sure to run npm install before running node main.js

In the previous article we learned about the basics of async.waterfall. If you missed it be sure to check it out here.

In this tutorial, we are going to look a practical example. Let's say you want to read the content of a file, replace the content, and then write to another file. If you were to do this nested callback, it would get a little bit hairy and hard to maintain and reason about. Let's see how we can do this with async.waterfall.

Before anything through, let's spend some time and learn about Node's conventions when it comes to callback functions. Given and object with a method that takes a callback function, the first argument of the callback is the error and the second holds the result of the operation:

object.method(params, function handleResult(err, result) {});

In the code above, the handleResult function takes two parameters: by convention the first argument holds any error that occurred during executing the object method. And the second argument, by convention holds the result of the method. So for example, if you were reading the content of a file with Node asynchronously, this is how your callback would look like:

fs.readFile('./file.txt', 'utf-8', function (err, content) {
  if (err) { return console.log(err);};
  console.log(content);
});

In this example, the callback function is function (err, content) {}. This function will be called when the content of the file has been read (we don't care when that is going to be). Once the file is read, this function will execute. Now, if there were any errors during the process of reading the file, they would be available through the first parameter of the callback function, that is the err parameter.

Now that we know about Node's convention, let's get started. As always, we need to setup our project:

mkdir -p ~/Desktop/read-write-file && cd $_ && touch main.js && npm init

Once you are prompted with the options, just accept the defaults. Once the package.json file is created, install async:

npm i async -S

Then open your main.js file of your project and add the following to the file:

var async = require('async');
var fs = require('fs');

async.waterfall([
    function readFile (done) {
      fs.readFile('./input.txt', 'utf-8', function (err, content) {
        done(err, content);
      });
    },
    function replaceContent (fileContent, done) {
      var newContent = fileContent.replace('world', 'there');
      done(null, newContent);
    },
    function writeToFile (toWrite, done) {
      fs.writeFile('./output.txt', toWrite, function (err, result) {
        done(err, result);
      });
    }
  ],
  function (theErr) {
    if (theErr) {
      throw new Error(theErr);
    } else {
      console.log('Done!');
    }
});

As you can see, with the first step function we read the content of the file and we pass the result to the next step function. In the second step function, we replace the text and pass the result to the third step function. And finally, we take the result of the second step function and simply write to file. You can then run this with node main.js and you should see that there is a new file created called output.txt that contains the string hello there!.

Now, if you remember in the beginning of this article we talked about Node's convention for callback functions. Because of that convention, we can simply pass the done parameter instead of explicitly creating callback functions. With that, our code will look much simpler:

async.waterfall([
    function readFile (done) {
      fs.readFile('./input.txt', 'utf-8', done); // <- just passing the parameter
    },
    function replaceContent (fileContent, done) {
      var newContent = fileContent.replace('world', 'there');
      done(null, newContent);
    },
    function writeToFile (toWrite, done) {
      fs.writeFile('./output.txt', toWrite, done); // <- just passing the done param
    }
  ],
  function (theErr) {
    if (theErr) {
      throw new Error(theErr);
    } else {
      console.log('Done!');
    }
});

I hope you found this tutorial useful. If you want to be notified about more tutorials like this, be sure to subscribe to the mailing list