Test runner

Version:

See buster-test

Module:

require("buster-test").testRunner;

In browsers:

buster.testRunner;

Evented test runner for both synchronous and asynchronous tests. The runner itself always executes asynchronously, making it very good at visualizing ongoing progress and helps avoid long running script warnings in browsers.

var testRunner = require("buster-test").testRunner;
var xUnitConsoleReporter = require("buster-test").xUnitConsoleReporter;

var runner = testRunner.create();
var reporter = xUnitConsoleReporter.create({ color: true });
reporter.listen(runner);

runner.runSuite([context, context2, ...]);

Events

suite:start event

Signature:

"suite:start", function () {}

Emitted once, as the runner starts running a test suite (typically when testRunner.runSuite() is called).

suite:end event

Signature:

"suite:end", function (results) {}

Emitted once, when all contexts are run (typically when testRunner.runSuite() completes).

context:start event

Signature:

"context:start", function (context) {}

Emitted every time a testContext object is entered.

context:end event

Signature:

"context:end", function (context) {}

Emitted every time a testContext object is completed.

context:unsupported event

Signature:

"context:unsupported", function (unsupported) {}

Emitted every time a context fails its requirements (when that happens, neither context:start or context:end are emitted).

test:setUp event

Signature:

"test:setUp", function (context) {}

Emitted once per test before the setup method(s) for a test is called.

test:start event

Signature:

"test:start", function (context) {}

Emitted after running the test’s setup(s), but before the test itself runs.

test:async event

Signature:

"test:async", function (test) {}

Emitted when a test has been found to be asynchronous (usually means that the test function was called and has returned).

test:tearDown event

Signature:

"test:tearDown", function (test) {}

Emitted once per test before the tear down method(s) for a test is called.

test:failure event

Signature:

"test:failure", function (error) {}

Emitted when the test throws (or otherwise flags) an AssertionFailure(). Only emitted once per test.

test:error event

Signature:

"test:error", function (error) {}

Emitted when the test throws any error that is not an AssertionFailure(). Only emitted once per test.

test:success event

Signature:

"test:success", function (test) {}

Emitted if the test passes.

test:timeout event

Signature:

"test:timeout", function (test) {}

Emitted if the test runner forcefully aborts the test. This happens when the test is asynchronous and does not resolve within the timeout configured by testRunnerOptions.timeout.

test:deferred event

Signature:

"test:deferred", function (test) {}

Emitted when a test is marked as deferred. The test is not run.

uncaughtException event

Signature:

"uncaughtException", function (exception) {}

Uncaught errors are errors that the test runner is unable to associate with the test that threw it. They occur in two situations:

• A synchronous test spawns an asynchronous task that results in an error. For instance, calling setTimeout() with a callback that throws an error in a synchronous test.
• An aborted asynchronous test throws (for instance, by failing an assertion).

The "uncaughtException" event will only be emitted when the environment supports it and the handleUncaughtExceptions property is set to true. Browsers that do not support window.onerror are unable to support this feature.

Methods

testRunner.create()

var runner = buster.testRunner.create([opts]);

Creates a new test runner instance.

testRunner.onCreate()

buster.testRunner.onCreate(function (runner) {});

Register a callback which is called everytime a runner is created with testRunner.create().

testRunner.runSuite()

runner.runSuite([context, context2, ...]);

Run an array of testContext object as a test suite.

testRunner.run()

runner.run(context);

Run a single testContext object. Note that this method does not trigger the suite:start event, and using it instead of testRunner.runSuite() may cause unintended behavior in Test reporters.

testRunner.assertionCount()

var count = runner.assertionCount();

The default implementation of this method is a no-op function. This method is called by the runner after each test to determine the number of assertions used in the test. It should not accumulate the assertion count.

Because the runner itself has no knowledge of the assertion library, this method is intended to be overridden by the assertion library in use. For instance, this is the integration necessary to count assertions with referee:

var assertions = 0;

buster.assert.on("pass", function () {
    assertions += 1;
});

buster.testRunner.onCreate(function (runner) {
    runner.on("test:start", function () {
        assertions = 0;
    });
});

buster.testRunner.assertionCount = function () {
    return assertions;
};

testRunner.assertionFailure()

runner.assertionFailure(exception);

Can be called from assertion libraries that do not throw an exception on assertion failure. For assertion failures to be picked up no matter what in asynchronous tests, this method needs to be called, as some exceptions are not possible for the runner to catch.

Properties

Test runner properties can be set when creating an instance, or simply by assigning to the property on an existing runner:

var runner = buster.testRunner.create({
    timeout: 500
});

// Or:

var runner = buster.testRunner.create();
runner.timeout = 500;

testRunnerOptions.failOnNoAssertions:

Default: true

When true, a test with no assertions is considered a failure. The number of assertions are measured with testRunner.assertionCount().

testRunnerOptions.timeout

Default: 250

When an asynchronous test runs for more than timeout ms, the runner will abort it and emit a test:timeout event.

testRunnerOptions.handleUncaughtExceptions

Default: true

When true, the runner will attempt to handle uncaught exceptions, by registering a listener on process for “uncaughtException” (Node.js) and assigning a callback to window.onerror (browsers).

Supporting objects

context object

See testContext object.

test object

See test object.

results object

A high-level numeric report. Emitted with suite:end event:

{
    contexts: 0,
    tests: 0,
    errors: 0,
    failures: 0,
    assertions: 0,
    timeouts: 0,
    deferred: 0
}

error object

An object representing a test failure (or error), emitted with test:failure event and test:error event:

{
    name: "Name of test",
    error: {
        name: "Type of exception",
        message: "Exception message",
        stack: "Stack trace as string"
    }
}

exception object

An exception-like object, emitted with uncaughtException event. In browsers, this object does not have a stack trace:

{
    name: "Type of exception",
    message: "Exception message",
    stack: "Stack trace as string"
}

unsupported object

Information about an unsupported context. Emitted with context:unsupported event. Contains an array of names of failed requirements and a context object:

{
    context: context,
    unsupported: ["label1", "label2"]
}