summaryrefslogtreecommitdiff
path: root/doc/test-suite.dox
blob: a7a55a2ef46e1dd3e1afb9d2c382d94321be14a5 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
/**
@page test-suite libinput test suite

libinput ships with a number of tests all run automatically on `ninja test`.
The primary test suite is the `libinput-test-suite-runner`. When testing,
the `libinput-test-suite-runner` should always be invoked to check for
behavior changes.

The test suite runner uses
[Check](http://check.sourceforge.net/doc/check_html/) underneath the hood
but most of the functionality is abstracted into *litest* wrappers.

The test suite runner has a make-like job control enabled by the `-j` or
`--jobs` flag and will fork off as many parallel processes as given by this
flag. The default if unspecified is 8. When debugging a specific test case
failure it is recommended to employ test filtures (see @ref test-filtering)
and disable parallel tests. The test suite automatically disables parallel
make when run in gdb.

@section test-config X.Org config to avoid interference

uinput devices created by the test suite are usually recognised by X as
input devices. All events sent through these devices will generate X events
and interfere with your desktop.

Copy the file `$srcdir/test/50-litest.conf` into your `/etc/X11/xorg.conf.d`
and restart X. This will ignore any litest devices and thus not interfere
with your desktop.

@section test-root Permissions required to run tests

Most tests require the creation of uinput devices and access to the
resulting `/dev/input/eventX` nodes. Some tests require temporary udev rules.
<b>This usually requires the tests to be run as root</b>. If not run as
root, the test suite runner will exit with status 77, interpreted as
"skipped" by ninja.

@section test-filtering Selective running of tests

litest's tests are grouped into test groups, test names and devices. A test
group is e.g.  "touchpad:tap" and incorporates all tapping-related tests for
touchpads. Each test function is (usually) run with one or more specific
devices. The `--list` commandline argument shows the list of suites and
tests. This is useful when trying to figure out if a specific test is
run for a device.

@verbatim
$ ./test/libinput-test-suite-runner --list
...
pointer:left-handed:
	pointer_left_handed_during_click_multiple_buttons:
		trackpoint
		ms-surface-cover
		mouse-wheelclickcount
		mouse-wheelclickangle
		low-dpi-mouse
		mouse-roccat
		mouse-wheel-tilt
		mouse
		logitech-trackball
		cyborg-rat
		magicmouse
	pointer_left_handed_during_click:
		trackpoint
		ms-surface-cover
		mouse-wheelclickcount
		mouse-wheelclickangle
		low-dpi-mouse
		mouse-roccat
		mouse-wheel-tilt
		mouse
		logitech-trackball
		cyborg-rat
		litest-magicmouse-device
	pointer_left_handed:
		trackpoint
		ms-surface-cover
		mouse-wheelclickcount
		mouse-wheelclickangle
		low-dpi-mouse
		mouse-roccat
		mouse-wheel-tilt
		mouse
...
@endverbatim

In the above example, the "pointer:left-handed" suite contains multiple
tests, e.g. "pointer_left_handed_during_click" (this is also the function
name of the test, making it easy to grep for). This particular test is run
for various devices including the trackpoint device and the magic mouse
device.

The "no device" entry signals that litest does not instantiate a uinput
device for a specific test (though the test itself may
instantiate one).

The `--filter-test` argument enables selective running of tests through
basic shell-style function name matching. For example:

@verbatim
$ ./test/libinput-test-suite-runner --filter-test="*1fg_tap*"
@endverbatim

The `--filter-device` argument enables selective running of tests through
basic shell-style device name matching. The device names matched are the
litest-specific shortnames, see the output of `--list`. For example:

@verbatim
$ ./test/libinput-test-suite-runner --filter-device="synaptics*"
@endverbatim

The `--filter-group` argument enables selective running of test groups
through basic shell-style test group matching. The test groups matched are
litest-specific test groups, see the output of `--list`. For example:

@verbatim
$ ./test/libinput-test-suite-runner --filter-group="touchpad:*hover*"
@endverbatim

The `--filter-device` and `--filter-group` arguments can be combined with
`--list` to show which groups and devices will be affected.

@section test-verbosity Controlling test output

Each test supports the `--verbose` commandline option to enable debugging
output, see libinput_log_set_priority() for details. The `LITEST_VERBOSE`
environment variable, if set, also enables verbose mode.

@verbatim
$ ./test/libinput-test-suite-runner --verbose
$ LITEST_VERBOSE=1 ninja test
@endverbatim

*/