Once the programmer has defined what the tests are these need to be run and the expected effects should be compared with the actual effects. FiveAM provides the function RUN for this purpose, RUN executes a number of tests and collects the results of each individual check into a list which is then returned. There are three types of test results: passed, failed and skipped, these are represented by TEST-RESULT objects.
Generally running a test will return normally, but there are two exceptional situations which can occur:
- An exception is signaled while running the test. If the variable *debug-on-error* is T than FiveAM will enter the debugger, otherwise a test failure (of type unexpected-test-failure) is returned. When entering the debugger two restarts are made available, one simply reruns the current test and another signals a test-failure and continues with the remaining tests.
- A circular dependency is detected. An error is signaled and a restart is made available which signals a test-skipped and continues with the remaining tests. This restart also sets the dependency status of the test to nil, so any tests which depend on this one (even if the dependency is not circular) will be skipped.
The functions RUN!, !, !! and !!! are convenient wrappers around RUN and EXPLAIN.
Variable *DEBUG-ON-ERROR* T if we should drop into a debugger on error, NIL otherwise.
Variable *DEBUG-ON-FAILURE* T if we should drop into a debugger on a failing check, NIL otherwise.
Variable *RUN-QUEUE* List of test waiting to be run.
(define-condition circular-dependency (error) ((test-case :initarg :test-case)) (:report (lambda (cd stream) (format stream "A circular dependency wes detected in ~S." (slot-value cd 'test-case)))) (:documentation "Condition signaled when a circular dependency between test-cases has been detected."))
Generic Function RUN-RESOLVING-DEPENDENCIES Given a dependency spec determine if the spec is satisfied or not, this will generally involve running other tests.
Method (RUN-RESOLVING-DEPENDENCIES TEST-CASE) Return true if this test, and its dependencies, are satisfied, NIL otherwise.
Method (RESOLVE-DEPENDENCIES SYMBOL) A test which depends on a symbol is interpreted as `(AND ,DEPENDS-ON).
Method (RESOLVE-DEPENDENCIES LIST) Return true if the dependency spec DEPENDS-ON is satisfied, nil otherwise.
Function RESULTS-STATUS Given a list of test results (generated while running a test) return true if all of the results are of type TEST-PASSED, faile otherwise.
Function RETURN-RESULT-LIST Run the test function TEST-LAMBDA and return a list of all test results generated, does not modify the special environment variable RESULT-LIST.
Generic Function %RUN Internal method for running a test.
Function RUN! Equivalent to (explain (run TEST-SPEC)).
Function EXPLAIN! Explain the results of RESULT-LIST using a detailed-text-explainer with output going to *test-dribble*
Function RUN Run the test specified by TEST-SPEC.
Function ! Rerun the most recently run test and explain the results.
Function !! Rerun the second most recently run test and explain the results.
Function !!! Rerun the third most recently run test and explain the results.