Unable to run the suites twice (with different configuration)

[sunrize531]

Currently it's impossible to run the tests twice. The second pass just claims that there are no suites to run.

Here is mochaTest config example:

module.exports = function (grunt) {
    "use strict";

    grunt.initConfig({
        mochaTest: {
            // Run tests
            tests: {
                options: {
                    captureFile: 'test-results.txt',
                    quiet: false,
                    timeout: 5000
                },
                src: ['test/**/test-*.js']
            },
            testsLegacy: {
                options: {
                    captureFile: 'test-results-legacy.txt',
                    quiet: false,
                    timeout: 2000
                },
                src: ['test/**/test-*.js']
            }
        }
    });


    grunt.loadNpmTasks('grunt-mocha-test');

    grunt.registerTask('tests', function () {
        process.env.NODE_ENV = '';
        grunt.task.run('mochaTest:tests');
    });

    grunt.registerTask('testsLegacy', function () {
        process.env.NODE_ENV = 'legacy';
        grunt.task.run('mochaTest:testsLegacy');
    });

    grunt.registerTask('testAll', function () {
        grunt.task.run(['tests', 'testsLegacy']);
    });
};

Here is the piece of grunt testAll output:

-> passed 50 of 50 tests (14718ms)

Running "testsLegacy" task
Running "mochaTest:testsLegacy" (mochaTest) task
-> 0 tests (1ms)

Done, without errors.

I believe it's related with this issue. Is it possible to do something like this, after all suites has been runned?

[pghalliday]

I'll take a look, but I'm not sure I understand the workaround. Also, usually, different tasks use different tests so reusing the mocha instance would break that use case. I don't think this will be trivial

[sunrize531]

I currently don't have enough time to look into code, unfortunately, but maybe the reason is that different tasks are indeed using the same mocha instance.

[pghalliday]

No, it's to do with the way mocha includes tests. It loads them using require, which results in Node putting them in the require cache. Thus the second Mocha instance does that, but as it's in the same Node process, any file previously loaded does not get loaded again and the tests don't get added. If you reuse the same mocha object then the tests are already loaded and you can run them again (possibly with different environment variables). As you can tell I had time to think about this and now understand how the workaround works.

You would still have issues with any environment dependent code run outside of a describe block as it would not run twice, but i think anything inside a callback would be run again. At the very least anything inside it, before, beforeEach, after, afterEach callbacks would be run (not sure about describe callbacks). If this sounds confusing then you can imagine what it will be like trying to debug such a problem if you run into it.

Still the problem remains - reusing the Mocha instance will break the use case of having different tests in different grunt tasks. I still have to think some more. I'm wondering if i can make it cache the instance for each task - so if you actually run the same task then it will use the same instance, but different tasks use different instances. You could then adjust the grunt config thus:

    grunt.registerTask('tests', function () {
        process.env.NODE_ENV = '';
        grunt.task.run('mochaTest:tests');
    });

    grunt.registerTask('testsLegacy', function () {
        process.env.NODE_ENV = 'legacy';
        grunt.task.run('mochaTest:tests');
    });

This assumes grunt doesn't get too clever and only run the task once, thinking it already did it. I don't think that will happen, but I can see that you will have issues specifying different options (ie. the capture file)

[sunrize531]

Unfortunately, both of those mocha tasks are part of my versioning task, and I definitely need them both in the same process, just to validate that my application supports legacy configuration before increasing the version number.