path: root/SummerOfCodeIdeas.mdwn

# Project Ideas for Google Summer of Code / X.Org Endless Vacation of Code programs


## Goal

The X.org board treats GSoC as an opportunity to teach new developers rather than a chance to get a pile of free code. With this perspective, if, in two months, the student actually has learned how to contribute to X Window System and its related projects, that's a huge step forward. Creating a project which guides this process with a maximal chance of success is the only tricky part.

When writing a proposal, please remember to make it detailed. Include at least the information called for in "[[What should a student proposal look like?|https://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2015/help_page#5._What_should_a_student_proposal_look]]", but including milestones and a project schedule is even better.  See [[GSoCApplication|GSoCApplication]] for guidelines.

X.Org is a large and comprehensive project with a huge number of possible opportunities for interesting Google / X.Org Summer of Code projects.  This list contains a few of those opportunities that are particularly interesting to X.Org developers and potential mentors. Please note that these are just suggestions; if you have an idea for something else please ask.

If you have questions, feel free to contact us on the [[dri-devel mailing list|http://lists.freedesktop.org/mailman/listinfo/dri-devel]] or the [[dri-devel IRC channel|irc://irc.freenode.net/#dri-devel]].

## 2017 Ideas

Here are a list of projects proposed by X.Org developers. If you have ideas of your own, please post them on the relevant mailing lists.

### Apitrace

#### Plug the core-apitrace support for performance counters to qapitrace

* _Difficulty:_ Medium
* _Skills Required:_ C++, Qt5
* _Helpful, but optional skills:_ CPU & GPU profiling
* _Where to ask questions:_ [[apitrace@lists.freedesktop.org|mailto:apitrace@lists.freedesktop.org]], #apitrace on irc.freenode.net
* _Description:_ Apitrace is an open source program that allows tracing, replaying, inspecting and profiling OpenGL/Direct3D calls made by any application. During the previous GSoC, gl_retrace received support for listing and monitoring performance counters, and an experimental GUI was developed. The goal of this GSoC will be to finish the GUI and get it merged in Apitrace. With this work done, this will allow game and driver developers to find bottlenecks and inefficiencies in their code and then tune their code to get more performance. Some necessary items:
   1. Study the design proposed by last year's student, possibly propose some changes
   1. Polish the GUI and land it upstream before the end of the project

Possible mentor: Martin Peres (mupuf on IRC/freenode)

#### In-place shader editing in qapitrace

* _Difficulty:_ Medium
* _Skills Required:_ C++, Qt5
* _Helpful, but optional skills:_ general OpenGL familiarity
* _Where to ask questions:_ [[apitrace@lists.freedesktop.org|mailto:apitrace@lists.freedesktop.org]], #apitrace on irc.freenode.net
* _Description:_ Apitrace is an open source program that allows tracing, replaying, inspecting and profiling OpenGL/Direct3D calls made by any application.  The project idea is to enable editing any of the shader traces in a glDrawXYZ state view in qapitrace, so that the user could 'Lookup State' again and see the results from the trace replayed with the modified shader.  Ie. qapitrace should edit the trace to insert the necessary GL calls to create the new shader and activate it for that particular draw.  (Possibly w/ a checkbox to apply globally to all draws that use that particular gl program?  In which case it would just edit the corresponding `glShaderSource()` call?)

Possible mentor: Rob Clark (robclark on IRC/freenode)

### Mesa

#### Switch OpenMAX state tracker in Mesa/Gallium to use Tizonia

* _Difficulty:_ Easy-Medium
* _Skills Required:_ C
* _Useful skills:_ Mesa, OpenMAX, gstreamer programming
* _Hardware/Software required:_ driver supported by the OpenMax state tracker (radeon, nouveau)
* _Where to ask questions:_ [[mesa-dev@lists.freedesktop.org|mailto:mesa-dev@lists.freedesktop.org]], #dri-devel on irc.freenode.net
* _Description:_
Currently the OpenMAX state tracker in Mesa/Gallium uses [[Bellagio|http://omxil.sourceforge.net/]]. The project would be to use [[Tizonia|https://github.com/tizonia/tizonia-openmax-il/]] instead. Bellagio has not been updated for a few years and only supports the OpenMAX IL 1.1 specification. Tizonia is actively maintained and also up to date with latest OpenMAX IL 1.2 specification.  Also Tizonia supports OMX_UseEGLImage which would allow for zero-copy decoding/rendering.  This work would benefit AMD and Nvidia hardware on Linux desktop. It would also help embedded developers by allowing them to develop feature on desktop first before to go to target.  Testing can be done through gst-omx, the set of GStreamer plugins that wrap OpenMAX.
    1. Add support for Tizonia in Gallium/st-omx. 
    1. Add zero copy support using OMX_UseEGLImage

Possible mentor: Julien Isorce

#### Implement remaining presentation features in the VDPAU state tracker

* _Difficulty:_ Easy-Medium
* _Skills Required:_ C
* _Useful skills:_ Mesa, image processing
* _Hardware/Software required:_ driver supported by the VDPAU state tracker (radeon, nouveau)
* _Where to ask questions:_ [[mesa-dev@lists.freedesktop.org|mailto:mesa-dev@lists.freedesktop.org]], #dri-devel on irc.freenode.net
* _Description:_
The VDPAU state tracker in Mesa still lacks some nice to have features, like lanczos scaling, inverse telecine, luma keying and a temporal spatial deinterlacer. A possible task would be to implement some or all of those and get them into a state where we can add them to the main Mesa project.

Possible mentor: Christian König

#### DriConf replacement

* _Difficulty:_ Medium
* _Skills Required:_ Writing advanced GUI
* _Useful skills:_ C/C++
* _Hardware/Software required:_ Any GPU working with Mesa
* _Where to ask questions:_ [[mesa-dev@lists.freedesktop.org|mailto:mesa-dev@lists.freedesktop.org]], #dri-devel on irc.freenode.net
* _Description:_ Mesa users can set several driver options via an xml file in $HOME/.drirc or system /etc/drirc. With these files, the user can define behaviours for selected applications, like forcing vsync, picking a specific gpu in multi-gpu systems or enabling driver workarounds. DriConf is an user-friendly GUI to set these options, but it is outdated (old GTK, not multi-gpu compatible, and missing several features). Missing a good GUI to visualize and set driver settings is one of the recurring complaints from Mesa users. The goal of this GSoC would be to write an advanced configuration tool for Mesa drivers.

Possible mentor: Nicolai Hähnle (nha on IRC/freenode)

### Freedreno (Open Source Adreno driver)

See also our [Trello board](https://trello.com/b/VC0IXzrq/freedreno).

#### Texture Tiling

* _Difficulty:_ Medium
* _Skills Required:_ C
* _Useful skills:_ 
* _Hardware/Software required:_ Adreno 3xx, 4xx, or 5xx GPU
* _Where to ask questions:_ [[freedreno@lists.freedesktop.org|mailto:freedreno@lists.freedesktop.org]], #freedreno on irc.freenode.net
* _Description:_ Add support for tiled textures. This will involve adding transfer handlers to perform the tiling, figuring out the various tiling options and when to use them, and to update all uses.

#### Compressed Textures

* _Difficulty:_ Medium
* _Skills Required:_ C
* _Useful skills:_ most of the work is in userspace (mesa), but some kernel experience wouldn't hurt, to also implement the display/scanout part
* _Hardware/Software required:_ Adreno 5xx GPU
* _Where to ask questions:_ [[freedreno@lists.freedesktop.org|mailto:freedreno@lists.freedesktop.org]], #freedreno on irc.freenode.net
* _Description:_ Starting with adreno 5xx, the hw supports rendering to and sampling (and scanning out) compressed textures to reduce memory bandwidth.  A bit more reverse engineering required about how this works for texture state, and how to do resolve blits.

#### Compiler: Spilling Registers
* _Difficulty:_ Difficult
* _Skills Required:_ C, Compilers
* _Hardware/Software required:_ Adreno 3xx, 4xx, or 5xx GPU
* _Where to ask questions:_ [[freedreno@lists.freedesktop.org|mailto:freedreno@lists.freedesktop.org]], #freedreno on irc.freenode.net
* _Description:_ Shaders with high register usage (see shadertoy) may fail register allocation stage, in which case load/store instructions need to be added to "spill" register values to memory.  This project would involve adding support to freedreno/ir3 compiler to determine which registers to spill, add necessary load/store instructions to save to local memory and restore from local memory (and then re-run scheduling and register allocation passes).

### Nouveau (Open Source NVIDIA driver)

A list of project is available on our [Trello board](https://trello.com/b/ZudRDiTL/nouveau).

#### Instruction Scheduler

* _Difficulty:_ Difficult
* _Skills Required:_ C++
* _Useful skills:_ Compilers
* _Hardware/Software required:_ NVIDIA Fermi or later
* _Where to ask questions:_ [[nouveau@lists.freedesktop.org|mailto:nouveau@lists.freedesktop.org]], #nouveau on irc.freenode.net
* _Description:_ Write an instruction scheduling pass for basic blocks in the nouveau codegen compiler. Create scheduling policies that improve performance.

#### Maxwell Accelerated Video Decoding

* _Difficulty:_ Medium
* _Skills Required:_ C
* _Useful skills:_ Reverse Engineering, Video formats
* _Hardware/Software required:_ NVIDIA Maxwell GM107 / GM108 chip
* _Where to ask questions:_ [[nouveau@lists.freedesktop.org|mailto:nouveau@lists.freedesktop.org]], #nouveau on irc.freenode.net
* _Description:_ RE the mechanism for video decoding in VP6 (the video engine iteration used in GM10x chips). Implement support for driving the engine.

#### Kepler Accelerated Video Encoding

* _Difficulty:_ Medium-Difficult
* _Skills Required:_ C
* _Useful skills:_ Reverse Engineering, Video formats
* _Hardware/Software required:_ NVIDIA Kepler GKxxx chip
* _Where to ask questions:_ [[nouveau@lists.freedesktop.org|mailto:nouveau@lists.freedesktop.org]], #nouveau on irc.freenode.net
* _Description:_ RE the video encoding component of Kepler chips and produce sufficient documentation for an open source implementation (while reusing blob firmware). A stretch goal would be to integrate something into nouveau to be able to drive the engine, and a toy user-space application to encode a video.

### Kernel

#### DRM Kernel Janitor

You may find multiple ideas from the the [DRM todo list](https://dri.freedesktop.org/docs/drm/gpu/todo.html) page.

### libinput

#### Support for pressure-only bluetooth styli

* _Difficulty:_ Medium
* _Skills Required:_ C
* _Useful skills:_ Reverse Engineering, knowledge about Bluetooth/bluez
* _Hardware/Software  required:_ Touchscreen-capable laptop, bluetooth pen
* _Where to ask questions:_ [[wayland-devel@lists.freedesktop.org|mailto:wayland-devel@lists.freedesktop.org]], #wayland on irc.freenode.net
* _Description:_ Pressure-senstive style are commonly used with iPads and tablets. These styli usually only provide pressure, relying on the touchscreen for coordinates. To present such a stylus to an application, the touchscreen data and the stylus data have to be combined to a single event from the stylus device. libinput is perfectly suited to do exactly that. This project requires work in bluez to implement the stylus-specific protocol as a bluez plugin so the device shows up as kernel device (and, depending on the device, reverse engineering the protocol). This project requires work in libinput to extract and combine the data from both devices into a single virtual device.

### Wayland

#### Multi-buffer Present in XWayland

* _Difficulty:_ Medium
* _Skills Required:_ C
* _Useful skills:_ Window system / display control infrastructure, tracing
* _Hardware/Software Required:_ Supports Wayland/Xorg with open drivers
* _Where to ask questions:_ [[wayland-devel@lists.freedesktop.org|wayland-devel@lists.freedesktop.org]], #wayland-devel on IRC
* _Description:_ XWayland as written only uses a single fixed buffer for each window/surface, both for software and hardware-accelerated rendering. This means that each buffer gets copied inside XWayland, and also the single-buffering means that tearing or incomplete content can be shown from clients. This project requires work inside XWayland in order to pass the client buffers directly to the host Wayland compositor, without copies or tearing.

#### Media recording protocol for Wayland

* _Difficulty:_ Medium
* _Skills Required:_ C
* _Useful skills:_ Media encode/decode fundamentals and API (GStreamer/VA-API/V4L2/...)
* _Hardware/Software Required:_ Supports Wayland and media encode (VA-API/V4L2) with open drivers
* _Where to ask questions:_ [[wayland-devel@lists.freedesktop.org|wayland-devel@lists.freedesktop.org]], #wayland-devel
* _Description:_ Screencasting in Wayland is currently limited to private compositor implementations, and not available to clients for screencasting (e.g. browsers). Whilst the compositor-private protocols can be extended, work would need to be done in order to negotiate an acceptable format with the clients, and integrate with hardware-accelerated media encode units. This project requires work inside a Wayland compositor (Weston, Mutter, KWin or Enlightenment) to provide a standard client-accessible protocol enabling screencasting of content in either a compressed/encoded, or raw, format.

#### Weston support for multiple KMS devices

* _Difficulty:_ Medium
* _Skills Required:_ C
* _Useful skills:_ KMS, Pixman/EGL
* _Hardware/Software Required:_ Supports Wayland and KMS with open drivers
* _Where to ask questions:_ 
* _Description:_ Weston supports multiple outputs, but currently limits them to a single GPU. Add support to Weston's DRM backend to open several KMS devices, with the ability to use outputs from all of them. To sidestep the problem of allocating buffers suitable for hardware acceleration across different GPUs, restrict this to the Pixman software renderer.

#### Improved Wayland timeline debugging

* _Difficulty:_ Medium
* _Skills Required:_ C
* _Useful skills:_ Performance profiling (including timeline and sample-based profilers), Wayland, toolkits, EGL / Vulkan WSI
* _Hardware/Software Required:_ Supports Wayland and KMS with open drivers
* _Where to ask questions:_ [[wayland-devel@lists.freedesktop.org|wayland-devel@lists.freedesktop.org]], #wayland-devel
* _Description:_ Weston currently has built-in support for timeline debugging, dumping JSON files with a visual representation of the timeline for showing client content on screen. This has a small client named Wesgr to render these to SVG. This project is somewhat open-ended, but there are many possibilities to improve it, including: implementing this protocol through more clients, a better renderer / timeline viewer with programmatic filtering (e.g. 'show me all frames which were late'), and integration with kprobes to instrument timings within the kernel stack (hardware/fence completion).

## Potential Mentors

Please add yourself below if you are willing to be a GSoC or EVoC mentor:

* [Rob Clark](mailto:robdclark@gmail.com) - Freedreno related projects, possibly general mesa or apitrace projects
* [Martin Peres](mailto:martin.peres@free.fr) - Apitrace, Power Management in Nouveau, xf86-video-modesetting or EzBench-related projects
* [Peter Hutterer](mailto:peter.hutterer@who-t.net) - anything input-related
* [Daniel Stone](mailto:daniel@fooishbar.org) - Wayland/Weston, EGL, KMS
* [Pekka Paalanen](mailto:pekka.paalanen@collabora.co.uk) - Wayland
* [Karol Herbst](mailto:karolherbst@gmail.com) - Nouveau