Running Tests¶
The main front-end for Slash is the slash run
utility, invoked from the command line. It has several interesting options worth mentioning.
By default, it receives the path to load and run tests from:
$ slash run /path/to/tests
Verbosity¶
Verbosity is increased with -v
and decreased with -q
. Those can be specified multiple times.
In addition to the verbosity itself, tracebacks which are displayed at the session summary can be controlled via tha --tb
flag, specifying the verbosity level of the tracebacks. 0
means no tracebacks, while 5
means the highest detail available.
See also
Loading Tests from Files¶
You can also read tests from file or files which contain paths to run. Whitespaces and lines beginning with a comment #
will be ignored:
$ slash run -f file1.txt -f file2.txt
Lines in suite files can optionally contain filters and repeat directive.
Filter allows restricting the tests actually loaded from them:
# my_suite_file.txt
# this is the first test file
/path/to/tests.py
# when running the following file, tests with "dangerous" in their name will not be loaded
/path/to/other_tests.py # filter: not dangerous
See also
The filter syntax is exactly like -k
described below
Repeat allows to repeat a line:
# my_suite_file.txt
# the next line will be repeated twice
/path/to/other_tests.py # repeat: 2
# you can use filter and repeat together
/path/to/other_tests.py # filter: not dangerous, repeat: 2
Debugging & Failures¶
Debugging is done with --pdb
, which invokes the best debugger available.
Stopping at the first unsuccessful test is done with the -x
flag.
See also
Including and Excluding Tests¶
The -k
flag to slash run
is a versatile way to include or exclude tests. Provide it with a substring to only run tests containing the substring in their names:
$ slash run -k substr /path/to/tests
Use not X
to exclude any test containing X in their names:
$ slash run -k 'not failing_' /path/to/tests
Or use a more complex expression involving or
and and
:
$ slash run -k 'not failing_ and components' /path/to/tests
The above will run all tests with components
in their name, but without failing_
in it.
Overriding Configuration¶
The -o
flag enables us to override specific paths in the configuration, properly converting their respective types:
$ slash run -o path.to.config.value=20 ...
See also
configuration
Running Interactively¶
As a part of the development cycle, it is often useful or even necessary to run your infrastructure in interactive mode. This allows users to experiment with your framework and learn how to use it.
Slash supports running interactively out-of-the-box, using the -i
flag to slash run
:
$ slash run -i
This will invoke an interactive IPython shell, initialized with your project’s environment (and, of course, a valid Slash session).
By default, the namespace in which the interactive test runs contains all content of the slash.g
global container. You can disable this behavior by setting interactive.expose_g_globals to False
.
Resuming Previous Sessions¶
When you run a session that fails, Slash automatically saves the tests intended to be run for later reference. For quickly retrying a previously failed session, skipping tests which had already passed, you can use slash resume
:
$ slash resume -vv <session id>
This command receives all flags which can be passed to slash run
, but receives an id of a previously run session for resuming. All unsuccessful tests are then rerun in a new session. You can control whether to attempt failed tests first or planned (not started) tests through the --failed-first
and --unstarted-first
command-line flags respectively. You can also control which tests to resume through the --failed-only
, --unstarted-only
, and --no-skipped
command-line flags. You can use combinations of the latter two but not with the first (failed-ony).
Rerunning Previous Sessions¶
You can rerun all the tests of a previous session, given the session’s tests were reported. This might be helpful when reproducing a run of specific worker, for example. You can use slash rerun
:
$ slash rerun -vv <session id>
This command receives all flags which can be passed to slash run
, but receives an id of a previously run session for rerunning.