This is part 1 of the JavaScript unit test framework series.

I have been involved in lots of agile and non-agile development projects, so I have experienced enough benefits of agile test-driven development to see a pattern. Smaller codebase, higher quality code, less bugs, more fun, lower startup threshold for adding new developers to the team, more efficient refactoring, maintainability, etc.

One key tool in achieving all of that is a unit test framework, which is why my first project is to develop such a framework in JavaScript. Also, I will use the very unit test framework I'm writing to unit-test the framework itself.

The intended users of this framework are developers, so I can use pretty technical language in the specs, but I will still focus on keeping requirements short and concise. The first group of requirements look like this:

• eng = new ut.Engine() should create a new unit testing engine
  • upon creating a new engine, the eng.testCount should be zero
• eng.add(name, testfunc) should add a new unit test to the engine
  • eng.tests[name] should point to the testfunc function
  • the eng.testCount property should be increased
• eng.run() should run the test function of each added unit test once
  • if running the test function throws an exception, that indicates a failed unit test
  • all unit tests should always run, even if some unit test function crashes (or even all of them)
• The engine should keep track of the number of failed/succeeded tests
  • after eng.run() is done, the eng.successCount and eng.failureCount should contain the number of succeeded/failed unit tests respectively, and the sum of them should be the same as eng.testCount

First requirement

After establishing this, writing and running the first test is easy:

// Tests

var engine = new ut.Engine();

engine.add("new ut.Engine should create a new unit testing engine",
    function() {
        // Act
        var eng = new ut.Engine();

        // Assert
        if (eng.testCount !== 0) {
            throw "Did not set testCount to zero";
        }
    });

engine.run();

Of course, when I try to run this, I will get a reference error saying "ut is not defined". Also, I won't be able to actually run any tests before both add and run are somewhat implemented, so here is iteration zero of ut.Engine:

// Implementation

var utEngine = function() {
    this.tests = [];
};

utEngine.prototype = {
    add : function(name, testfunc) {
        this.tests.push(testfunc);
    },

    run : function() {
        for (var i = 0; i < this.tests.length; ++i) {
            var testfunc = this.tests[i];
            testfunc.call();
        }
    }
};

window.ut = {
    "Engine" : utEngine
};

Now it's possible to add and run unit tests, so it actually produces the first failing unit test output, albeit only visible in the developer console: "Did not set testCount to zero". One small code change, and no errors are thrown:

// Implementation

var utEngine = function() {
    this.tests = [];
    this.testCount = 0;
};

Second requirement

The next requirement deals with adding test functions to the tests collection and increasing the testCount property. This is what those tests look like:

// Tests

engine.add("add() should set tests[name] to func",
    function() {
        // Arrange
        var eng = new ut.Engine();
        var bar = function() {};

        // Act
        eng.add("foo", bar);

        // Assert
        if (eng.tests["foo"] !== bar) {
            throw "tests.foo does not point to bar";
        }
    });

engine.add("add() should increase testCount",
    function() {
        // Arrange
        var eng = new ut.Engine();
        var func = function() {};

        // Act
        eng.add("foo", func);

        // Assert
        if (eng.testCount !== 1) {
            throw "Did not increase testCount";
        }
    });

The first test is made to pass by refactoring the tests array into an anonymous object, changing the add method to add by name instead of by index, and the run method to traverse the object using for ... in. The second test passes after a small refactoring of the add method:

// Implementation

var utEngine = function() {
    this.tests = {};
    this.testCount = 0;
};

utEngine.prototype = {
    add : function(name, testfunc) {
        this.tests[name] = testfunc;
        ++this.testCount;
    },

    run : function() {
        for (var name in this.tests) {
            var testfunc = this.tests[name];
            testfunc.call();
        }
    }
};

Third requirement

If this one isn't satisfied, any failing test will stop all concurrent tests from running, which will only allow us to deal with one failing test at a time.

// Tests

engine.add("run() should run each added test once",
    function() {
        // Arrange
        var eng = new ut.Engine();
        var called = [0, 0];
        var func1 = function() { called[0]++; };
        var func2 = function() { called[1]++; };
        eng.add("func1", func1);
        eng.add("func2", func2);

        // Act
        eng.run();

        // Assert
        if (called[0] !== 1) {
            throw "Did not call func1";
        }
        if (called[1] !== 1) {
            throw "Did not call func2";
        }
    });

engine.add("run() should run all tests even when crash",
    function() {
        // Arrange
        var eng = new ut.Engine();
        var called = 0;
        var func = function() {
            ++called;
            throw "Crash!";
        }
        eng.add("Going once", func);
        eng.add("Going twice", func);

        // Act
        eng.run();

        // Assert
        if (called !== 2) {
            throw "Did not call both added tests";
        }
    });

The first test of this requirement already passes, but the second one does crash. It doesn't even run all the way to the assertion part. It's the test function itself that produces the developer console output – not the unit test framework. To make the test pass, I simply wrap calling test functions in try ... catch.

// Implementation

run : function() {
    for (var name in this.tests) {
        var testfunc = this.tests[name];
        try {
            testfunc.call();
        } catch(e) { }
    }
}

Fourth requirement

When writing the unit test for this requirement, I also adopt the new way of checking for failing unit tests. Instead of just letting the developer console print out the exception message, I print out the values of failureCount, successCount and testCount after run() has been called.

// Tests

engine.add("The engine should count successes and failures",
    function() {
        // Arrange
        var eng = new ut.Engine();
        var failFunc = function() {
            throw "Crash!";
        };
        var successFunc = function() { };
        eng.add("One fail", failFunc);
        eng.add("Two fails", failFunc);
        eng.add("Three fails", failFunc);
        eng.add("One success", successFunc);

        // Act
        eng.run();

        // Assert
        if (eng.successCount !== 1) {
            throw "successCount should be 1, but is " + eng.successCount;
        }
        if (eng.failureCount !== 3) {
            throw "failureCount should be 3, but is " + eng.failureCount;
        }
    });

engine.run();

console.log(engine.failureCount + " failures and " +
            engine.successCount + " successes, out of " +
            engine.testCount + " tests");

When running all unit tests now, the output simply says "undefined failures and undefined successes, out of 6 tests", simply because the engine does not yet count failures or successes. A small refactoring of the constructor and the run method later:

// Implementation

var utEngine = function() {
    this.tests = {};
    this.testCount =
        this.successCount =
        this.failureCount = 0;
};

// inside prototype definition:
run : function() {
    for (var name in this.tests) {
        var testfunc = this.tests[name];
        try {
            testfunc.call();
            ++this.successCount;
        } catch(e) {
            ++this.failureCount;
        }
    }
}

There it is – the first iteration of my JavaScript unit test framework. Feel free to use it. There is still lots of important stuff to do, like a way of knowing which tests are failing and not just how many of them.

The latest version of the code is always available in the GitHub repository.

Articles in this series:

• JavaScript unit test framework, part 1 of 4 (this part)
• JavaScript unit test framework, part 2 of 4
• JavaScript unit test framework, part 3 of 4
• JavaScript unit test framework, part 4 of 4