- MOOSE provides an extendable Test Harness for executing your code with different input files.
- Each kernel (or logical set of kernels) you write should have test cases that verify the code's correctness.
- The test system is very flexible. You can perform many different operations, for example: testing for expected error conditions.
- Additionally, you can create your own "Tester" classes which extend the Test Harness.
- Related tests should be grouped into an individual directory and have a consistent naming convention.
- We recommend organizing tests in a hierarchy similar to your application source (i.e. kernels, BCs, materials, etc).
- Tests are found dynamically by matching patterns (highlighted below)
my_kernel_test.e [input mesh]
my_kernel_test.i [input file]
tests [test specification file]
gold/ [gold standard folder - validated solution]
A quick look at the test specification file
- Same format as the standard MOOSE input file
type = Exodiff
input = my_kernel_test.i
exodiff = my_kernel_test_out.e
type = RunException
input = my_kernel_exception.i
expect_err = 'Bad stuff happened with variable \w+'
Testers provided in MOOSE
- RunApp: Runs a MOOSE-based application with specified options.
- Exodiff: Checks Exodus files for differences within specified tolerances.
- CSVDiff: Checks CSV files for differences within specified tolerances.
- RunException: Tests for various error conditions.
- CheckFiles: Checks for the existence of specific files after a completed run.
Adding Additional Testers (Advanced)
- Inherit from
Tester and override:
- prepare(): Method to run right before a test starts
- getCommand(): Command to run (in parallel with other tests)
- processResults(): Process the results to check whether the test has passed
- NO REGISTRATION REQUIRED!
- Drop the
Tester object in "[HTML_REMOVED]/scripts/TestHarness/testers"
Options available to each Tester
- input: The name of the input file
- exodiff: The list of output filenames to compare
- abs_zero: Absolute zero tolerance for exodiff
- rel_err: Relative error tolerance for exodiff
- prereq: Name of the test that needs to complete before running this test
- min_parallel: Minimum number of processors to use for a test (default: 1)
- max_parallel: Maximum number of processors to use for a test
- . . .
Running your tests
|run ‘n' jobs at a time
|run tests in debug mode (debug binary)
|run just one set of tests
|run regular tests and those marked ‘heavy'
|write separate log file for each failed test
|all the tests in a user defined group
|quiet (don't print output of FAILED tests
|request to run each test with ‘n' procs
Other Notes on Tests
- Individual tests should run relatively quickly ($$$\sim$$$2 second rule)
- Outputs or other generated files should not be checked into the subversion repository.
- Do not check in the solution that is created in the test directory when running the test!
- The MOOSE developers rely on application tests when refactoring to verify correctness
- Poor test coverage = Higher code failure rate