diff options
Diffstat (limited to 'DriMemoryManagerDesign.mdwn')
| -rw-r--r-- | DriMemoryManagerDesign.mdwn | 256 |
1 files changed, 256 insertions, 0 deletions
diff --git a/DriMemoryManagerDesign.mdwn b/DriMemoryManagerDesign.mdwn new file mode 100644 index 0000000..7db4978 --- /dev/null +++ b/DriMemoryManagerDesign.mdwn @@ -0,0 +1,256 @@ + +_**DRI Memory Manager Design**_ + +**Contents:** [[!toc ]] + + +## Introduction + +The memory manager is roughly divided into two portions. There is a small portion in the kernel, and the remainder resides in the user-mode driver. This division is made for two reasons that are worth noting here. By having a majority of the memory manager in user-mode avoids calling into the kernel for each memory operation. On recent Linux kernels the cost of calling into the kernel has been greatly reduced, so this isn't as much of an issue as it once was. + +There is bigger issue driving this design decision. The division of code between kernel and user roughly matches the division of mechanism and policy. All of the policy decisions regarding which memory is allocated are made in user-mode. + + +## Memory Manager Interface + +The userspace portion of the memory manager will live in libdrm. This means we can abstract the inner-working of the memory manager described below from the interface offered to X.org DDX and Mesa DRI drivers. This interface is open for discussion. + +Feel very free to change these descriptions (Dave Airlie). + + +### drmMMCreate + + +[[!format txt """ +void *drmMMCreate(int fd); +"""]] +Create a instance of the DRM memory manager for the DRM open on the file descriptor fd. Before calling this the DRM driver must have been bootstrapped and the pools created by the DRM. This function just initialises all the memory manager userspace structures needed. + + +### drmMMDestroy + + +[[!format txt """ +void drmMMDestroy(void *mm_handle); +"""]] +Destroy an instance of the DRM memory manager. Free any internal structures allocated. + + +### drmMMAllocRegion + + +[[!format txt """ +#define DRM_MM_ATTRIB_PINNED 0x1 +#define DRM_MM_ATTRIB_ADDMAP 0x2 +#define DRM_MM_ATTRIB_BACKED 0x4 + +drm_region_id_t drmMMAllocRegion(void *mm_handle, int pool_id, unsigned int size, int alignment, unsigned int attributes, void *backing); +"""]] +Allocate a region for this process from the pool identified by pool_id, of size size aligned with alignment. + +Pinned attribute makes the memory unmovable, this may need to be a privileged operation or not, scanout buffers need to be pinned. Addmap attribute will cause the DRM to add a mapping for this block, (not sure if useful just a suggestion) - may need to be pinned. Backed attribute means the client is providing a block of memory to back the area via the *backing pointer. - again suggestion only. + +Memory used by the driver as DMA or vertex buffers will have to be pinned, at least for the current usage where commands can be placed in DMA buffers without the lock held. Any memory passed by the driver to the client will also have to be pinned, at least for as long as the client has access to it. Once the DMA buffer is enqueued for execution, or retrieved from the client, the pinned attribute can be removed. (So we'll need a call to do that?) + + +### drmMMFreeRegion + + +[[!format txt """ +void drmMMFreeRegion(void *mm_handle, drm_region_id_t region_id) +"""]] +Free the region of memory specified by region_id. + + +### drmMMGetRegionPointer + + +[[!format txt """ +void *drmMMGetRegionPointer(void *mm_handle, drm_region_id_t region_id) +"""]] +Return a pointer to the actual memory behind the region. (These pointers should not be stored across lock drops). + + +### drmMMGetHandleMappedRegion + + +[[!format txt """ +drm_handle_t drmMMGetHandleMappedRegion(void *mm_handle, drm_region_id_t region_id); +"""]] +Returns a DRM handle for an area that has been allocated using the ADDMAP attribute. + + +### drmMMMSetRegionAttributes + + +[[!format txt """ +int drmMMSetRegionAttributes(void *mm_handle, drm_region_id_t region_id, unsigned int new_attribs); +"""]] +Set the new attributes on the region. + + +### drmMMGetRegionAttributes + + +[[!format txt """ +unsigned int drmMMGetRegionAttributes(void *mm_handle, drm_region_id_t region_id) +"""]] +Return the current attributes of a region. + + +### drmMMSetRegionFence + + +[[!format txt """ +int drmMMSetRegionFence(void *mm_handle, drm_region_id_t region_id, unsigned int condition); +"""]] +Set the condition of a fence on the region specified by region_id. (Do multiple fences on a region make any sense?? or does the only last fence setting matter...) + + +### drmMMTestFence + + +[[!format txt """ +int drmMMTestRegionFence(void *mm_handle, drm_region_id_t region_id); +"""]] +Test the fence on a region_id. + + +### drmMMWaitFence + + +[[!format txt """ +int drmMMWaitRegionFence(void *mm_handle, drm_region_id_t region_id); +"""]] +Wait until the region fence is completed. (Unsure if this is needed - do we really want to wait in the drmMM layer - maybe needs a higher level interface). + + +## Memory Manager Concepts + +The memory manager is based around the concept of allocating a block of memory to a named object. The object and the associated block of memory is called a _region._ Each region has an associated ID. This ID is a simple unsigned integer that is unique _per-device._ + +By allocating memory to a named object, the owner of that object can be notified when portions of that object have been removed from memory by another process. For example, a process can allocate memory for a texture object. Another process can later over-write part of that object with its own object. Before the first process can reuse that texture, it needs to be sure that all of the texture is still in memory. + +For objects that cannot just be "dumped" from memory, such as buffers used as rendering targets, a region of _backing storage_ can be attached. These regions are called _precious._ When another process needs to reclaim a block of memory from a precious region, it must first copy the data out of the region into the backing store. + +Changes to the state of memory are track via a log. As a process allocates memory, frees memory, or changes the attributes of a region, it writes that information to a circular buffer in shared memory. Each process independently processes the data in the log to keep its local view of the memory state up to date. In distributed computing, this is a concept known as [[virtual synchrony|http://en.wikipedia.org/wiki/Virtual_synchrony]]. + + +## Kernel Interface + +The interface to the kernel portion of the memory manager consists of only five ioctls. These ioctls are used to query parameters about memory pools, allocate region IDs, release region IDs, configure backing store for regions, and backing up regions to their backing store. Of these five, only three will actually be implemented in the first phase of the implementation. + + +### Bootstrapping Memory Pools + +There is no direct user / kernel interface for bootstrapping the kernel portion of the memory manager. Rather than providing interfaces for a user-mode controller process (e.g., the X-server) to configure the available memory pools, this task is handled by the device-specific bootstrap process. For example, in the MGA DRM code, the function mga_do_dma_bootstrap (in [[mga_dma.c|http://cvs.freedesktop.org/dri/drm/shared-core/mga_dma.c?view=markup]]) knows what memory pools are available (e.g., on-card and AGP for AGP cards and on-card for PCI cards), how big they are, and why types of data can be stored in them. This is _exactly_ the information that is needed to bootstrap the memory manager. Since the kernel already has all the required information, there is no reason to add new interfaces. + +Once the device-dependent code has initialized all of the memory pools that are available, it must create the logs. The log data is stored in a series of pages in a shared memory area. The first page describes where the log data can be found in the remaining pages. The format of this data is an array of structures. The array is indexed by pool ID. + + +[[!format txt """ +struct drm_mmgr_pool_log_descriptor { + unsigned int first_page; /**< First page containing log data. */ + unsigned int page_count; /**< Number of pages containing log data. */ +}; +"""]] + +### Management of Region IDs + +Region IDs are allocated from the DRM via the `DRM_IOCTL_MMGR_ALLOC_IDS` ioctl. Region IDs are given to processes in fairly large, fixed size blocks. Internally region IDs are tracked by a two page bitmap. On common processors, this 65,536 region ID blocks. Since a region ID consists of 30-bits, each block represents 16,384 IDs. The first ID allocated and the count of IDs allocated is returned in the `drm_mmgr_region_id_range_t` structure by the `DRM_IOCTL_MMGR_ALLOC_IDS` ioctl. + + +[[!format txt """ +typedef struct drm_mmgr_region_id_range { + unsigned int base_id; /**< First region ID. */ + unsigned int count; /**< Number of IDs. */ +} drm_mmgr_region_id_range_t; +"""]] +Once a block of IDs has been allocated to a process, it is the responsibility of the device-specific DRM code to track. When the process closes the DRM file handle, all ID blocks used by the process must be released. + +Processes can also release blocks of IDs by calling the `DRM_IOCTL_MMGR_RELEASE_IDS` ioctl. Since IDs are always allocated in fixed size blocks, only the base ID of the block being released is passed to this ioctl. + + +### Querying Pool Values + +There are two types off data about the memory manager that user-mode can query from the kernel. Both types of query are done via the `DRM_IOCTL_MMGR_GETPARAM` ioctl. User-mode can query information about per-device memory manager state and per-pool memory manager state. + +All `DRM_IOCTL_MMGR_GETPARAM` queries use the `drm_mmgr_getparam_t` structure. For per-device queries the `pool_id` field is not used and should be set to zero. The size and type of data pointed to by `value` varies with the query. + + +[[!format txt """ +typedef struct drm_mmgr_getparam { + /** + * Name of the parameter. See \c drm_mmgr_getparam_mode_t. + */ + unsigned int param; + + unsigned int pool_id; /**< ID of the pool being queried. */ + void __user * value; /**< Storage for the value being queried. */ +} drm_mmgr_getparam_t; +"""]] +A typical bootstrap procedure is for the user-mode device driver to query `DRM_MMGR_PARAM_NUM_POOLS` for the device, then query `DRM_MMGR_PARAM_POOL_IDS`. The initial `DRM_MMGR_PARAM_NUM_POOLS` query is required so that the driver knows how much data will be returned in `value` by the `DRM_MMGR_PARAM_POOL_IDS` query. The pool ID values are unique within a system. That is, if a system has four graphics cards the each memory pool will have an ID that is unique across all cards. + +Once the IDs of the available pools are known, the size (via `DRM_MMGR_PARAM_POOL_SIZE`) and attribute bits (via `DRM_MMGR_PARAM_POOL_ATTRIBUTES`) are queried. + +The initial state of pool memory (i.e., which portions of memory are allocated or free) are queried via the `DRM_MMGR_PARAM_POOL_STATE_SIZE` / `DRM_MMGR_PARAM_POOL_STATE`. The former is used to determine the amount of data, measured in bytes, returned by the later. `DRM_MMGR_PARAM_POOL_STATE` returns an array of `drm_mmgr_puddle_data_t` structures in `value`. This array is then used to initialize the driver's internal representation of memory. `DRM_MMGR_PARAM_POOL_STATE_SIZE` must always return an integer multiple of `sizeof( drm_mmgr_puddle_data_t )`. + + +[[!format txt """ +typedef struct drm_mmgr_puddle_data { + unsigned int size; /**< Size, in bytes, of the puddle. */ + unsigned int region_id; /**< Region ID owning the puddle. A region + * ID of zero means that the puddle is not + * allocated. + */ +} drm_mmgr_puddle_data_t; +"""]] +The final bootstrap step is for the user-mode driver to get a handle for the pool's logs. The handle for the logs and size of the logs are queried via `DRM_MMGR_PARAM_LOG_HANDLE` and `DRM_MMGR_PARAM_LOG_SIZE`, respectively. `DRM_MMGR_PARAM_LOG_SIZE` is measured in bytes but must be an integer multiple of the system's page size. The first page is an array, indexed by pool ID, of `drm_mmgr_log_header_t` structures. The remaining pages make up the log data. The log data for each pool will always begin at a page boundary. The log data will also always have a size that is a power of two. + + +[[!format txt """ +typedef struct drm_mmgr_log_header { + unsigned int size; /**< Size, in bytes, of the pool's log. */ + unsigned int offset; /**< Offset, in bytes, from the start of the + * log data pages to the pool's log. + */ + volatile uint64_t index; /**< Index to next position in log. */ + + /** + * Last index value processed by the kernel. + */ + volatile uint64_t kernel_index; +} drm_mmgr_log_header_t; +"""]] + +### The Log + +The log isn't strictly a kernel interface, but it is created by and obtained from the kernel. However, the kernel does 'not' write data to the log. Each process maintains a local 64-bit counter representing the last index into the log that it has processed. Before accessing a region, the process must compare its local counter with the `drm_mmgr_log_header_t::index` for the pool containing the region. If the values are not equal, the process much "play back" instructions from the log until they are equal. + +Each instruction in the log begins with an `unsigned int` header containing two pieces of data. The least-significant 2-bits of the value specify the instruction, and the next 30-bits contain the associated region ID. On architectures where `unsigned int` is more than 32-bits, the most-significant bits must be zero. Depending on the instruction, the initial `unsigned int` may be followed by one or more additional `unsigned int` fields. + +When accessing the log, the index `current_index & (log->size - 1)` is used. If the value of `log->index & ~(log->size - 1)` is greater than `current_index & ~(log->size - 1)` and the value of `log->index & (log->size - 1)` is greater than `current_index & (log->size - 1)` the log has wrapped around since the last time the process played back log data. This means that the process has missed some instructions. This means that the process must re-bootstrap its view of memory by querying `DRM_MMGR_PARAM_POOL_STATE_SIZE` and `DRM_MMGR_PARAM_POOL_STATE`. + +A process can also determine if a memory operation will wrap the log for the kernel. Since the kernel's view of memory is used to bootstrap (and re-bootstrap) new processes, it is critical that the log **never** wraps for the kernel. By performing a similar comparison using `log->kernel_index` and `current_index` the process can determine if a wrap is about to occur. If the log is about to wrap for the kernel, the process can prevent this cataclysmic event by querying `DRM_MMGR_PARAM_POOL_STATE_SIZE`. In order to calculate the value for this query the kernel must play the entire log. Making this query can trick the kernel into catching up thereby avoiding a wrap. + + +#### DRM_MMGR_LOG_ALLOCATE + +This instruction is used to allocate or free a block of memory. If the region ID is non-zero, the instruction allocates a block of memory and associates it with the specified region ID. If the region ID is zero, the instruction releases a block of memory. Immediately following the initial `unsigned int` are two additional `unsigned int` fields. The first specifies the offset of the memory block, and the second specifies the size of the memory block. + + +#### DRM_MMGR_LOG_SET_ATTRIBUTE + +This instruction is used to specify the attribute bits associated with a region ID. It is invalid to specify a region ID of zero with this instruction. No attribute bits are currently specified. In the (near) future bits for "precious" or "pin" will be specified. It is also possible that additional "hints", such as "region fragmented" or "region won't be used again", can be created. Beyond a few common attributes the remaining bits will be available for device-specific uses. + + +#### DRM_MMGR_LOG_SET_FENCE + +This instruction is used to specify the attribute bits associated with a region ID. It is invalid to specify a region ID of zero with this instruction. Fences are similar in concept to the texture aging mechanism used in most open-source DRI drivers. The DRM for each device maintains a monotonically increasing counter. This counter increases when the hardware completes some operation. Some hardware automatically maintains this counter, whereas other hardware notifies the driver to increment the counter via an interrupt. The mechanism by which the counter is incremented is not important. The important issue is that the user-mode driver be able to predict what the value of this counter will be when the hardware is finished using a region. The predicted value is then set as the _fence_ for that region. If the current counter is greater than or equal to the fence, then the hardware is finished using that region. + +By communicating these fence values in the log, the user-mode drivers can correctly order memory operations. Some cards (e.g., Radeon cards) can pipeline texture uploads. On cards with this ability can safely ignore the fence value for texture allocations. However, even on cards that can pipeline texture uploads other data, such as secondary command buffers, are not pipelined. A block of memory allocated for one of these regions must not contain a region with a pending fence. + + +### Backing Objects to Main Memory + +To be completed. |