Multi tool use

Question

When I run all our JUnit tests, using eclipse, can I set a default timeout?

Background

My manager insists on writing Unit tests that sometimes take up to 5 minutes to complete. When I try to run our entire test suite (only about 300 tests) it can take over 30 minutes. I want to put something in place that will stop any test that takes longer than 10 seconds.

I know an individual test can be annotated with:

@Test(timeout=10000)


But doing this would make his long tests always fail. I want them to work when he runs them on his box (if I have to make minor adjustments to the project before checking it in, that's acceptable. However, deleting the timeouts from 40 different test files is not practical).

I also know I can create an ant task to set a default timeout for all tests, along the lines of:

<junit timeout="10000">
...
</junit>


The problem with that we typically run our tests from inside eclipse with Right Click > Run As > JUnit Test.

Summary

So is there a relatively painless way to set a timeout for all tests, perhaps using a Run Configuration setting, or project setting, or JUnit preference, or environment variable, or something? I'd even settle for installing some other plugin that lets me right click on particular test folders and run all the tests in some other manner like through ant or something...

@test(timeout=10)

@Test(timeout=10)

7 Answers
7

Possible solution:
Extend all your Test classes from another class: TestBase for example

TestBase

Add to TestBase global timeout. This timeout will be applied to all extended classes:

TestBase

public class TestBase {
@Rule
public Timeout globalTimeout = new Timeout(10000);
}


@Test(timeout=10)

So maybe a combination of using Infinitest with the "Slow test warning" enabled together with the filtering feature would do the trick. You could identify tests that exceed your time-limit and add them to the filter list, this would only affect testing from inside Eclipse. Running the tests via a possible build script via CLI/CI etc would not be affected at all.
You can find more on setting this up here: http://improvingworks.com/products/infinitest/infinitest-user-guide/

If you want to configure the tests to run for a maximum of ten seconds you can try this:

@Test(timeout=10000)

@Test(timeout=10000)

doing this would make his long tests always fail. I want them to work when he runs them on his box . . . deleting the timeouts from 40 different test files [before checkin] is not practical

My manager insists on writing Unit tests that sometimes take up to 5 minutes to complete

This almost certainly indicates that those tests are not in fact unit tests. Cut that Gordian knot: try refactoring your testsuite to provide equivalent test coverage without requiring a test-case that runs for that long.

Clock

MockClock

Clock

SystemClock

Clock

Almost certainly your bosses tests are system tests pretending to be unit tests. If they are suppsoed to be unit tests and are just slow they should be refactored to use mocks so that they run quicker.

Anyway, a more pragmatic and diplomatic approach than confronting your boss over this might be to just try and run the faster ones yourself. I've seen a hack to do this in a project where slow tests had SytemTest in their names. Then there were two ant targets created in the build file. One that ran all tests and one that filtered out by class name the SytemTests . To implement this all you would have to do is rename some of the tests and write your ant target.

SytemTest

SytemTest

It sounds like test suites would help you out.

You can have two test suites; QuickTests and AllTests. Include QuickTests in the AllTests suite, as well as the tests that take a long time. Then all other tests would go into the quick tests suite.

QuickTests

AllTests

From eclipse you can run an entire test suite at once. So you would run QuickTests and that way all the other slow tests will not run.

Or see this question on how to apply a timeout to a suite which will apply to nested suites and classes in the suite. Which can achieve similar to what you want when combined with my above suggestion.

I know this doesnt really answer your question but the simple answer is don't!

Setting timeouts conditionally is wrong beacsue then you would have Unit tests on your machine that you are always going to fail. The point of Unit tests is to be able to quickly see that you havnt broken anything. Having to check through the failed test list to make sure it's just the long running tests is ust going to make some bugs pass through the cracks.

As some of the commenters mentioned you should split out the tests into unit tests that run quickly and the slower runnning integration tests i.e. have a source folder called src/main/java for your code, src/test/java for unit tests and src/integration-test/java for the longer running tests.

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.