1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
|
# User-space Input Driver Architecture
While most of the current input drivers are focused around X.org, other system applications that deal with input hardware often lack proper input support. Especially upcoming wayland compositors need a uniform way to parse input data to provide it to their clients. The existing X.org *xf86-input-\** drivers are tightly bound to X.org and hard to re-use by other X11-independent applications. Furthermore, there is a broad consensus that we should try to decouple input-drivers and compositors and provide a universal independent input system instead.
This page explain how this input system could look like. Any feedback and criticism is welcome and everyone is invited to help improving it.
## Objectives
The whole architecture tries to follow these objectives:
* *Independent* of any graphics API and system (X11, Wayland, ...)
* **No** unnecessary *background daemons*
* Use *existing infrastructure*
* Allow kernel-space and **user-space** device *I/O drivers*
* Allow dynamic input-driver configuration
* Allow dynamic input-driver addition and removal
* Support server/client systems, but also client-only
* Do not require CAP_SYS_ADMIN or root rights
* provide **machine-writable** and **user-writable** configuration
## Overview
The input-architecture is built around a central compositor. The compositor needs access to input-devices, both for its own use and optionally to pass input data on to connected clients. Whether the compositor is a server/client architecture or a client-only application does not matter. However, the compositor is expected to be a privileged system-daemon that is allowed to access input resources.
The input-architecture is responsible of:
* enumerating existing input hardware
* monitoring the system for input hardware hotplug-events
* analyzing input hardware and detecting device capabilities
* reading/writing input events from/to the hardware
* parsing input/output events and providing standardized data to the compositor
* configuring input devices (even during runtime) according to the user's needs
### Event/Data Flow
[<img src="http://people.freedesktop.org/~dvdhrm/architecture_basic.png" width="600px" height="338px" class="img" align="right" alt="Input Architecture Overview" />](http://people.freedesktop.org/~dvdhrm/architecture_basic.png)
The image on the right shows the basic event and data flow. The compositor is responsible of setting up **udev_monitor** objects to monitor the system for hotplug-events of input devices. With **udev_enumerate** it can get a list of currently available input devices. The udev-system is out-of-scope of this document and is an already established interface. However, the information that is provided by udev and **udev_device** (or via sysfs) is crucial to understand this architecture. Moreover, the reader is expected to be familiar with udev and sysfs.
On the compositor's side a device is always uniquely identifiable by the sysfs path of the corresponding input device (**not** the evdev child device!). A compositor uses **libevdev** to open the evdev character device node and retrieves information on the device. libevdev hides the raw kernel API behind a well-defined user-space API and provides several helpers to avoid duplicating buffer-management (eg., **SYN_DROPPED** handling) and event-mapping in every compositor.
After a device has been reported by udev and opened via libevdev, a compositor queries **libdevdb** (formerly **libinputmapper**) to retrieve additional information and (user-)configuration data for the given device. Specific sections of this data-block can be parsed by the compositor for generic device handling. Other sections are ignored by the compositor and may be used by device drivers or client applications to retrieve further information. libdevdb provides an extensive set for dynamically extendable configuration options. Instead of providing configuration-file parsers in every application of the input-stack, we move configuration handling to a central place (which is libdevdb).
One very important part of the data-block provided by libdevdb is a list of device-tags. These tags describe what capabilities are supported by a device and which drivers may be loaded on it. A compositor uses these tags to decide which device-driver to load for the device. After the device driver is fully setup, clients may optionally be notified that a new device was detected. By passing the sysfs-path of the device or a special libdevdb ID to the client, a client can query libdevdb for additional information that weren't of interest to the compositor.
Once a device is initialized, a compositor starts the input-event stream. It uses **libevdev** to read events from the kernel evdev device and immediately forwards them to the loaded device-driver to update its internal state. The device-driver signals to the compositor whether any driver-specific event is available. This event is then handled by the compositor in an application-specific manner or forwarded to the clients. Output events (eg., force-feedback) is handled the other way around. An output event is generated by a client (or by the compositor itself) and forwarded to the loaded device-driver. The device driver updates internal state and optionally passes raw evdev output-events to the compositor. The compositor forwards them untouched to libevdev which then writes them back to the kernel evdev device.
## In-depth View
Following an in-depth view on each part of the input architecture. Some of these parts are already well-established and used by other user-space systems. Some other parts are tailored specifically for this input architecture.
### Device-enumeration and hotplugging (UDev)
TODO: Describe how device-detection works (especially multi-seat support)
### Device detection and configuration (libdevdb)
**libdevdb** (formerly libinputmapper) is the central configuration and detection library. It is responsible for telling a compositor what to do with a given device and how to configure it. As input data it gets the sysfs path of the input device. It collects different resources (eg., device-name, VID/PID, capabilities, ...) and then matches the device on its database. If matched, it returns the information to the caller.
Note that despite its name, the library is expected to use a huge set of statically compiled heuristics to perform generic matching, instead of using a huge database for each and every device. However, for device-specific quirks or user-specific configuration, a broad range of configuration files is supported.
The first-level configuration consists of huge "match" files. These files contain matching-rules against the different resources that are available for each device (eg., device-name, kernel-driver-name, VID/PID, ...). Each matching-rule then points to a directory which contains the rule files to apply on any matched device:
# global match-file
[match "Fix Wii-Remote Mappings"]
tags = gamepad,joystick
name = Nintendo Wii Remote*
rules = ./wiimote.d/
force = -gamepad,-jostick
[match "Custom keyboard layout"]
tags = keyboard
vid = 0x3667
pid = 0x0100
rules = ./custom/my_keyboard.rule
The "tags" matching-rule allows to match against the TAGs that were already applied by the heuristics of libdevdb. These heuristics are applied before matching against the database. The "rules" instruction contains a path to a file or directory of custom rule files (described below) that are to be applied on matched devices. The "force" instruction allows overwriting the tags that were detected by the heuristics. In the example above, the "gamepad" and "joystick" tags are removed for Wii-Remotes as the generic drivers cannot handle such devices (they report custom data).
The rules-files, on the other hand, provide custom configuration. Lets extend the example above:
# My Keyboard Rules (./custom/my_keyboard.rule)
[libxkbcommon]
layout = de
variant = nodeadkeys
This rule file, according to the match-rule above, is loaded on a specific custom keyboard and provides a section that can be parsed by *libxkbcommon*. Other drivers ignore this section, as does the compositor. Only libxkbcommon parses this section.
Any driver or client may use these files as global data-storage. However, note that this data is static. It is meant as user-configuration, not as dynamic database. However, settings daemons can easily provide match-files and rule-files in ~/.config/devdb/ that are machine-written. So gnome-settings-daemon could write these files if a user modifies the keyboard layout via the GUI.
### Kernel Evdev Event/Device handling (libevdev)
[libevdev](http://github.com/whot/libevdev) is a relatively new library that hides the complexity of the kernel's evdev API. It caches device-information to avoid calling system-calls/ioctls more than once. It hides buffer management by providing **SYN_DROPPED** handling and can refresh device state at any time.
TODO: Describe in more detail what libevdev does.
### Device Drivers
Every input-device type needs a specific device-driver to parse specific events. The driver typically has two interfaces. The first interface takes raw input events and returns raw output events. This interface is used to connect an input-driver with **libevdev** and, thus, with the input hardware. The second interface provides the parsed device data. This interface is defined by the device-driver itself and must be supported by the compositor to be used. For instance, this interface provides *KEY_UP* and *KEY_DOWN* events for keyboards together with a set of currently depressed keyboard modifiers. But a compositor needs to explicitly know how to handle keyboards to make use of this API. This second interface can also be used to send events to the hardware. For example, a joystick driver may provide a sophisticated force-feedback API and the device-driver converts it into raw hardware-events that are forwarded to the kernel.
[<img src="http://people.freedesktop.org/~dvdhrm/driver_multiple_backends.png" width="600px" height="338px" class="img" align="right" alt="Multiple Backends per Driver" />](http://people.freedesktop.org/~dvdhrm/driver_multiple_backends.png)
Generally speaking, there are two different types of device drivers. There are drivers like libxkbcommon which take a well known set of linux **input_event** objects, updates its internal keyboard state and returns a well known set of XKB events. Then there is the other set of drivers which converts custom input events to linux **input_event** events, which can then be handled by the first type drivers. For example, imagine there is a *foobar* input-device with a camera. And it returns absolute position events according to your current finger-position. If you project a keyboard-layout at a wall and scan it with this camera, you can use it as a virtual keyboard. However, the input from the device is absolute position data which libxkbcommon can make no use of. But you also want to avoid duplicating libxkbcommon. So what you do is writing a **libfoobarcommon** device-driver which takes absolute position events and converts it to linux **input_event** keyboard events. These events can then be passed straight into libxkbcommon by the compositor and you will get useful keyboard events from this device.
This driver-stacking requires minimal compositor support but allows a very modular infrastructure. A real existing use-case is the Nintendo Wii-Remote Gyroscope+Accelerometer input, that can be converted to relative mouse-motion by xf86-input-xwiimote. If we would allow loading a libmousecommon on the output of this hypothetical libwiimotecommon driver, we could reuse mouse-acceleration and more from libmousecommon and keep libwiimotecommon small.
### The Compositor's Job
Independent of device detection and configuration, a compositor has a dynamic input-driver system. This allows loading input-drivers dynamically instead of having hard-coded support for each of them. While input-drivers with specific interfaces (like touchpad, mouse, XKB, ...) need compositor-support, input-drivers with generic interfaces (that is, chain-loaded drivers that return raw "input_event" objects) can be loaded at any time. With each driver, the compositor notes down which devices each driver can deal with. It uses libdevdb device-tags and assigns a driver to each tag. If multiple drivers support the same tag, the one with highest priority is used over the others.
A compositor uses udev to query the hardware database. This is a standard procedure that the reader should be familiar with. For each input device that it finds, it creates a virtual device-object. This virtual device is identified by the sysfs path of the input device. Via *libevdev* the corresponding evdev char-device is opened and the device is initialized. After that *libdevdb* is queried to return the device-configuration and device-tags.
If no driver is registered for a given tag, the device is ignored. Otherwise, the driver with the highest priority on the given tag is loaded on the device. If a device has multiple tags, one driver for each tag is loaded.
TODO: Explain in more detail what a compositor needs to do.
### Clients
Client-side device handling obviously depends on the protocol that is used. The simplest and most straightforward way would be passing all data straight to the client without loading any device driver on it. The client would have to load any libfoobarcommon driver and parse the evdev events that it got from the compositor. We could pass the **libdevdb** IDs to the client so it can retrieve device-configurations from it.
Most notably, this approach breaks any feedback path. We cannot allow the client direct kernel evdev access, but we also cannot stupidly forward any output requests from the client to the device. So this approach, while very simple, only serves well as fallback for unsupported devices.
For other devices, we load a device-driver in the compositor and add a device-specific protocol for the client. This is the same that we do for keyboards. We parse the data in the server and forward it to the client as XKB data.
#### Client-side libdevdb Access
As we discussed above, libdevdb provides a static device configuration database that can be useful for clients, too. For instance, we could save information on gamepad geometry in these files and interested clients could read it to print proper user-prompts instead of the default "press button A". So a compositor passes the path to the currently loaded rules-directories of this device to the client. The client can then use libdevdb to parse these rules and retrieve requested information.
To avoid duplicating dnotify instances on each client, we simply require a compositor to notify a client whenever the configuration of the device changed.
## TODO
* Support user-space I/O drivers alongside kernel evdev devices. The current architecture requires user-space I/O drivers to either use **uinput** to provide a kernel evdev interface or hide the logic in the device-driver itself. It is questionable whether an extended interface is needed to avoid uinput overhead, but we should at least consider it. Drivers which could use this are: Input drivers via IEEE-802.11 (eg., Wireless-Display input devices), custom Bluetooth input devices, ...
## Current State
* udev device detection is implemented and works
* libevdev is being worked on: [libevdev on github](https://github.com/whot/libevdev/)
* libdevdb is still TODO
* libxkbcommon is available and working
* no other driver is currently available
## Contributors
* David Herrmann <dh.herrmann> (gmail)
|
