The TestHarness is the Python code behind the
run_tests script in MOOSE and every MOOSE-based application. It is responsible for finding tests and running them. Here we describe how to use the
run_tests script and how to create your own tests.
The ideas behind testing are described over the in the MOOSE Test System documentation.
run_tests is a small Python script that can be found in every MOOSE-based application and in the framework itself (under
run_tests script will find tests and run them with your compiled binary for your app.
To run the tests for any application, make sure the executable is built (generally by running
make see the Build System) and then do:
./run_tests -j 8
There are many options for
run_tests, but the
-j option shown above is by far the most widely used. It tells the
run_tests script how many processors to utilize on your local machine for running tests concurrently. Put the correct number there (instead of 8). The script will then go find and run your tests and display the results.
To see more options for
run_tests you can invoke it with
-h. There are many options to look through, but some of the important ones are:
--failed-tests: Runs the tests that just failed. Note: as long as you keep using the
--failed-testsoption the set of failed tests will not change.
--n-threads #: Causes the tests to run with
#of (OpenMP/Pthread/TBB) threads.
-p #: Causes the tests to run with
#MPI processes. Useful for guaranteeing that you get the same result in parallel!
--valgrind: Run all of the tests with the Valgrind utility. Does extensive memory checking to catch segfaults and uninitialized memory errors.
--recover: Run all of the tests in "recovery" mode. This runs the test halfway, stops, then attempts to "recover" it and run until the end. This is very rigorous testing that tests whether everything in your application can work with restart/recover.
testroot file is a small configuration file that is formatted like a MOOSE input file. It is read by the
run_tests script. It should be placed in the root of your application directory (i.e. right next to where the binary is). Some things you can set in that file are:
app_name: A unique, short name for your application.
trueby default, set this to
falseto make all warnings from running tests be _errors_ instead.
trueby default if set to
falsethen syntax errors in your test input files will be treated as errors.
The one thing we do not recommend is enforcing that the use of deprecated code should be treated like an error. That is entirely too rigid of a requirement and impedes the normal flow of development. Instead, developers should periodically run their tests with
--error-deprecated to see if any of their tests are using deprecated code / parameters and then fix them up at that point. The MOOSE team is not responsible for fixing deprecations.