WIP: open source userspace documentationwip-open-source-userspace

1 files changed, 90 insertions, 0 deletions

diff --git a/Documentation/process/open-source-driver-userspace.rst b/Documentation/process/open-source-driver-userspace.rst
new file mode 100644
index 000000000000..da64fcc946ff
--- /dev/null
+++ b/Documentation/process/open-source-driver-userspace.rst
@@ -0,0 +1,90 @@
+Open-Source Userspace Requirements
+==================================
+
+Introduction
+============
+
+Many kernel subsystems interact with user space applications via standard interfaces such as block devices or network devices. However there a lot of subsystem domains where the "device driver" contains a kernel and a userspace part, and the two co-exist to provide user applications with services.
+
+For these subsystems it is impractical to create a standard kernel interface, as the application interfaces are much higher up, so trying to force a standard uapi is a variant of the midlayer mistake. We don't agree to having stable ABIs in the kernel internals for binary modules, creating kernel interfaces for binary-only userspace is making the same mistake, just further up the stack.
+
+The primary example of this is GPUs. However FPGAs, DSPs and ML accelerators are starting to become more common in places where Linux is used, and operate along the same model.
+
+The DRM subsystem through many years of experience has strict requirements on introducing new uAPI. This document is an effort to generalise those requirements to cover more subsystems of the kernel where maintainers agree.
+
+
+Position Statement
+==================
+
+Any addition of subsystem device uAPI requires corresponding open-sourced userspace patches, and those patches must be reviewed and ready for merging into a suitable and canonical open source upstream project.
+
+It would be preferred the open source project is a vendor neutral collaborative environment, or that vendors would be encouraged to work on such a thing in advance of upstreaming kernel drivers.
+
+Reasoning
+=========
+
+Subsystems covered by this position are usually dealing with complex pieces of hardware. Userspace and kernel have to work together really closely.  The interfaces must be extremely wide and flexible, and therefore it is almost always impossible to precisely define them for every possible corner case. This in turn makes it practically infeasible to differentiate between behaviour that's required by userspace, and which must not be changed to avoid regressions, and behaviour which is only an accidental artifact of the current implementation.
+
+Without access to the full source code of all userspace users that means it
+becomes impossible to change the implementation details, since userspace could
+depend upon the accidental behaviour of the current implementation in minute
+details. Debugging such regressions without access to source code is pretty
+much impossible. As a consequence this means:
+
+- The Linux kernel's "no regression" policy holds in practice only for
+  open-source userspaces. Developers are perfectly fine if closed-source blob drivers in userspace use the same uAPI as the open drivers, but they must do so  in the exact same way as the open drivers. Creative (ab)use of the interfaces will, and in the past routinely has, lead to breakage.
+
+- Any new userspace interface must have an open-source implementation as
+  demonstration vehicle.
+
+The other reason for requiring open-source userspace is uAPI review. Since the
+kernel and userspace parts of the stack must work together so closely, code
+review can only assess whether a new interface achieves its goals by looking at
+both sides. Making sure that the interface indeed covers the use-case fully
+leads to a few additional requirements:
+
+- The open-source userspace must not be a toy/test application, but the real
+  thing. Specifically it needs to handle all the usual error and corner cases.
+  These are often the places where new uAPI falls apart and hence essential to
+  assess the fitness of a proposed interface. The vendor may have a second
+  optimised userspace stack, but the open-source once must be capable of
+  exercising all aspects of the hardware and uAPI.
+
+- The open-source userspace only extends to what is considered the next level
+  programming API. Think OpenGL, Vulkan, Level0. Not what is built on top of
+  these.
+
+- The userspace side must be fully reviewed and tested to the standards of that
+  userspace project. Each subsystem should establish their own rules around the
+  requirements in this situation.
+
+- The userspace patches must be against the canonical upstream, not some vendor
+  fork. This is to make sure that no one cheats on the review and testing
+  requirements by doing a quick fork.
+
+- The kernel patch can only be merged after all the above requirements are met,
+  but it **must** be merged to a subsystem next tree **before** the
+  userspace patches land. uAPI always flows from the kernel, doing things the
+  other way round risks divergence of the uAPI definitions and header files.
+
+
+More Background
+===============
+
+Reasons found over the years:
+- uAPI moves fast for new devices, testing older interfaces work in the way
+  userspace expects is important. This means having access to the userspace
+  in a testing environment.
+- long term maintenance of uAPI requires being able to identify and fix
+  security and other bugs in it. Doing this without knowing how the userspace
+  uses it can be close to impossible without references.
+- Designing common inter-driver/subsystem interfaces is hard, getting locking
+  right is hard, getting locking in inter-driver/subsystem interfaces with
+  possible userspace binaries interacting with the hardware started to get
+  really difficult. To reason about these you need a full stack view, from
+  the userspace hw access to the ISA of the device.
+- common userspaces create environments for common vendor projects to flourish.
+  Why give up the advantage of Linux being a vendor neutral project and
+  community just to let userspace stacks run rampant.
+- stabilising the uAPI for a binary userspace makes the mid layer mistake of
+  thinking you can design that will not limit future open source userspace