summaryrefslogtreecommitdiff
path: root/src/gallium/winsys/amdgpu/drm/amdgpu_cs.c
AgeCommit message (Collapse)AuthorFilesLines
2024-02-17winsys/amdgpu: convert amdgpu_cs.c to .cppMarek Olšák1-1863/+0
it will use a C++ template Acked-and-Tested-by: Leo Liu <leo.liu@amd.com> Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27627>
2024-02-07winsys/amdgpu: don't ref/unref slab BOs in amdgpu_cs_submit_ibMarek Olšák1-16/+31
It's pointless to increase the refcount of the backing BO and then decrease it in the same function when we already reference the slab entry BO that holds the reference of the backing BO. amdgpu_do_add_buffer is inlined in amdgpu_cs_submit_ib, so the new parameter doesn't cost anything. Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27408>
2024-02-07winsys/amdgpu: simplify amdgpu_do_add_buffer to remove memsetMarek Olšák1-5/+3
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27408>
2024-02-07winsys/amdgpu: skip code checking RADEON_USAGE_SYNCHRONIZED for slabsMarek Olšák1-6/+12
We unset RADEON_USAGE_SYNCHRONIZED, but we still checked it. Iterate over such buffers separately without checking RADEON_USAGE_SYNCHRONIZED. Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27408>
2024-02-07winsys/amdgpu: merge 2 loops iterating over real BOs in amdgpu_cs_submit_ibMarek Olšák1-24/+28
The first loop updates sequence numbers. The second loop creates the BO list. Do both in the same loop. Each loop has to reload the whole BO list from the L2 cache or higher, so we do it twice. By merging the loops, we only load the BO list from the L2 cache once. The final result is actually 2 loops, but they iterate over different ranges of the BO list, so each element is read only once. If global_bo_list is enabled, it overwrites the BO list after all is done. Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27408>
2024-02-07winsys/amdgpu: merge 2 loops iterating over sparse BOs in amdgpu_cs_submit_ibMarek Olšák1-64/+48
The first loop is inside amdgpu_add_sparse_backing_buffers. The second loop iterates over sparse BOs to update sequence numbers. Each loop has to reload the whole BO list from the L2 cache or higher, so we do it twice. By merging the loops, we only load the BO list from the L2 cache once. Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27408>
2024-02-07winsys/amdgpu: merge 2 loops iterating over slab entries in amdgpu_cs_submit_ibMarek Olšák1-9/+30
The first loop is inside amdgpu_add_slab_backing_buffers. The second loop iterates over BOs to update sequence numbers. Each loop has to reload the whole BO list from the L2 cache or higher, so we do it twice. By merging the loops, we only load the BO list from the L2 cache once. Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27408>
2024-02-07winsys/amdgpu: move code out of amdgpu_add_bo_fences_to_dependencies for reuseMarek Olšák1-11/+22
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27408>
2024-02-07winsys/amdgpu: change the signature of amdgpu_add_bo_fences_to_dependenciesMarek Olšák1-6/+5
for future work Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27408>
2024-02-07winsys/amdgpu: add real buffers of slab entries in the CS threadMarek Olšák1-39/+36
This removes the overhead from the driver thread and simplifies amdgpu_cs_add_buffer. Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27408>
2024-02-07winsys/amdgpu: fix a race condition when reading ws->num_buffersMarek Olšák1-1/+1
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27408>
2024-01-26winsys/amdgpu: use syncobj rather than amdgpu fencePierre-Eric Pelloux-Prayer1-101/+41
syncobjs provide the same features and allow to unify code paths because we don't need to handle imported syncobj separately. This simplifies the code and doesn't seem to have any perf impact. Syncobjs are supported in amdgpu since kernel commit 660e855813f78 during 4.12 cycle but the minor version wasn't bumped so use the next bump value asthe minimum supported version. Reviewed-by: Marek Olšák <marek.olsak@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24724>
2024-01-25winsys/amdgpu: Fix build on BSDFriedrich Vock1-0/+10
Some BSDs don't define ENODATA. Fixes: cf24d155 ("winsys/amdgpu: Hook up guilt to amdgpu_ctx_set_sw_reset_status") Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27181>
2024-01-18winsys/amdgpu: Limit usage of query_reset_state2Joshua Ashton1-42/+40
Following discussion on kernel mailing list[1], we are not gaining anything from using this to figure out if we reset, and it does not handle soft recovery. We will hear about the context loss and rationale when we submit. Instead, only use this for figuring out if the reset we already knew about was completed. [1]: https://lists.freedesktop.org/archives/amd-gfx/2024-January/103337.html Signed-off-by: Joshua Ashton <joshua@froggi.es> Reviewed-by: André Almeida <andrealmeid@igalia.com> Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Reviewed-by: Marek Olšák <marek.olsak@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27097>
2024-01-18winsys/amdgpu: Hook up guilt to amdgpu_ctx_set_sw_reset_statusJoshua Ashton1-3/+16
Certain return values have certain meanings indicating context loss and guilt. Hook those up here. Signed-off-by: Joshua Ashton <joshua@froggi.es> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27097>
2024-01-15winsys/amdgpu: cosmetic touchups around IB sizesMarek Olšák1-12/+14
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27060>
2024-01-15winsys/amdgpu: represent max_ib_size_dw in bytesMarek Olšák1-8/+7
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27060>
2024-01-15winsys/amdgpu: represent IB_MAX_SUBMIT_DWORDS in bytesMarek Olšák1-3/+3
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27060>
2024-01-15winsys/amdgpu: remove misplaced duplicated commentMarek Olšák1-4/+0
The same comment is in the header file where it should be. Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27060>
2024-01-15winsys/amdgpu: don't clear fence list elements after IB submissionMarek Olšák1-5/+2
Same idea as with the buffer lists. Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27060>
2024-01-15winsys/amdgpu: add more fence_reference helpersMarek Olšák1-0/+11
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27060>
2024-01-15winsys/amdgpu: don't clear buffer list elements after IB submissionMarek Olšák1-4/+4
amdgpu_winsys_bo_reference(dst, NULL) sets *dst to NULL, but we never read *dst again because we set num_buffers to 0. So don't touch the buffer list elements and only decrement the BO refcount. It makes a difference when you have 10k BOs. The CS thread overhead in VP2020/Catia1: 9.23% -> 8.74% Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27060>
2024-01-15winsys/amdgpu: cosmetic touchupsMarek Olšák1-5/+8
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27060>
2024-01-15winsys/amdgpu: merge loops decrementing num_active_ioctls & unreferencing bufsMarek Olšák1-11/+23
We have 2 separate loops doing that. num_active_ioctls was decremented in amdgpu_cs_submit_ib, while buffers were unreferenced in amdgpu_cs_context_cleanup immediately after it. Split unreferencing buffers from amdgpu_cs_context_cleanup, so that it's not done in amdgpu_cs_submit_ib, which will do it in the same loop where num_active_ioctls is unreferenced. Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27060>
2024-01-10winsys/amdgpu: remove amdgpu_bo_real::gpu_address, use amdgpu_va_get_start_addrMarek Olšák1-1/+2
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26914>
2024-01-06winsys/amdgpu: simplify code using amdgpu_cs_context::chunk_ibMarek Olšák1-59/+19
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06winsys/amdgpu: don't use amdgpu_fence::ctx for fence dependenciesMarek Olšák1-1/+4
The only remaining use of ctx is amdgpu_fence_is_syncobj. Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06winsys/amdgpu: use pipe_reference for amdgpu_ctx refcountingMarek Olšák1-4/+5
this is the standard utility for refcounting Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06winsys/amdgpu: implement explicit fence dependencies as sequence numbersMarek Olšák1-14/+18
This eliminates redundant fence dependencies if BOs add the same ones. Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06winsys/amdgpu: remove dependency_flags parameter from cs_add_fence_dependencyMarek Olšák1-2/+1
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06winsys/amdgpu: rename amdgpu_bo_sparse::lock -> commit_lockMarek Olšák1-3/+3
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06winsys/amdgpu: rewrite BO fence tracking by adding a new queue fence systemMarek Olšák1-104/+140
This decreases the time spent in amdgpu_cs_submit_ib from 15.4% to 8.3% in VP2020/Catia1, which is a decrease of CPU load for that thread by 46%. Overall, it increases performance by a small number in CPU-bound benchmarks. The biggest improvement I have seen is VP2020/Catia2, where it increases FPS by 12%. It no longer stores pipe_fence_handle references inside amdgpu_winsys_bo. The idea is to have a global fixed list of queues (only 1 queue per IP for now) where each queue generates its own sequence numbers (generated by the winsys, not the kernel). Each queue also has a ring of fences. The sequence numbers are used as indices into the ring of fences, which is how sequence numbers are converted to fences. With that, each BO only has to keep a list of sequence numbers, 1 for each queue. The maximum number of queues is set to 6. Since the system can handle integer wraparounds of sequence numbers correctly, we only need 16-bit sequence numbers in BOs to have accurate busyness tracking. Thus, each BO uses only 12 bytes to represent all its fences for all queues. There is also a 1-byte bitmask saying which sequence numbers are initialized. amdgpu_winsys.h contains the complete description. It has several limitations that exist to minimize the memory footprint and updating of BO fences. Acked-by: Yogesh Mohan Marimuthu <yogesh.mohanmarimuthu@amd.com> Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06r300,r600,radeonsi: switch to pb_buffer_leanMarek Olšák1-8/+8
to remove pb_buffer::vtbl from all buffer structures Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06gallium/pb_buffer: define pb_buffer_lean without vtbl, inherit it by pb_bufferMarek Olšák1-4/+4
amdgpu doesn't need vtbl. Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06winsys/amdgpu: remove va (gpu_address) from amdgpu_bo_slab_entryMarek Olšák1-4/+5
Keep it only in amdgpu_bo_real and amdgpu_bo_sparse. Slab entries can compute it from the slab BO and adding their entry index. Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06winsys/amdgpu: remove now-redundant amdgpu_bo_slab_entry::realMarek Olšák1-1/+1
The pb_slab pointer can be used to get the BO pointer because pb_slab is inside the BO structure now. Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06radeon_winsys: add struct radeon_winsys* parameter into fence_referenceMarek Olšák1-1/+8
Since the radeon winsys implements fences as buffers, we need radeon_winsys* to destroy them. This will enable the removal of pb_cache_entry::mgr. Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06winsys/amdgpu: rename to amdgpu_bo_slab to amdgpu_bo_slab_entryMarek Olšák1-5/+5
It's a slab entry. "Slab" is the whole buffer, which is AMDGPU_BO_REAL if we want to be precise. Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2023-12-20radeonsi/winsys: add cs_get_ip_type functionPierre-Eric Pelloux-Prayer1-0/+8
Will be used in the next commit. Reviewed-by: Marek Olšák <marek.olsak@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26774>
2023-12-13winsys/amdgpu: fix amdgpu_cs_has_user_fence for VPEMarek Olšák1-6/+3
invert the condition to prevent this in the future. Reviewed-by: Yogesh Mohan Marimuthu <yogesh.mohanmarimuthu@amd.com> Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26547>
2023-12-13winsys/amdgpu: clean up the rest of the code for cs->buffer_listsMarek Olšák1-39/+34
This just changes how we iterate over the buffer lists. Reviewed-by: Yogesh Mohan Marimuthu <yogesh.mohanmarimuthu@amd.com> Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26547>
2023-12-13winsys/amdgpu: pass amdgpu_buffer_list* to amdgpu_add_bo_fences_to_dependenciesMarek Olšák1-12/+6
Reviewed-by: Yogesh Mohan Marimuthu <yogesh.mohanmarimuthu@amd.com> Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26547>
2023-12-13winsys/amdgpu: return amdgpu_cs_buffer* from add/lookup_buffer instead of indexMarek Olšák1-38/+40
because we only use the index to get amdgpu_cs_buffer. Reviewed-by: Yogesh Mohan Marimuthu <yogesh.mohanmarimuthu@amd.com> Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26547>
2023-12-13winsys/amdgpu: clean up duplicated code around amdgpu_lookup/add_bufferMarek Olšák1-114/+25
The function for real, slab, and sparse buffers can be unified, mostly. Reviewed-by: Yogesh Mohan Marimuthu <yogesh.mohanmarimuthu@amd.com> Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26547>
2023-12-13winsys/amdgpu: change amdgpu_lookup_buffer to take struct amdgpu_buffer_listMarek Olšák1-24/+11
Reviewed-by: Yogesh Mohan Marimuthu <yogesh.mohanmarimuthu@amd.com> Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26547>
2023-12-13winsys/amdgpu: change real/slab/sparse_buffers to buffer_lists[3]Marek Olšák1-76/+97
The next commits will simplify it. Reviewed-by: Yogesh Mohan Marimuthu <yogesh.mohanmarimuthu@amd.com> Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26547>
2023-12-13winsys/amdgpu: don't count memory usage because it's unusedMarek Olšák1-34/+7
Acked-by: Yogesh Mohan Marimuthu <yogesh.mohanmarimuthu@amd.com> Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26547>
2023-12-13winsys/amdgpu: move lock from amdgpu_winsys_bo into sparse and real BOsMarek Olšák1-4/+4
The slab BO doesn't use it. Reviewed-by: Yogesh Mohan Marimuthu <yogesh.mohanmarimuthu@amd.com> Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26547>
2023-12-13winsys/amdgpu: use inheritance for the slab BOMarek Olšák1-1/+1
Reviewed-by: Yogesh Mohan Marimuthu <yogesh.mohanmarimuthu@amd.com> Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26547>
2023-12-13winsys/amdgpu: use inheritance for the sparse BOMarek Olšák1-2/+2
Reviewed-by: Yogesh Mohan Marimuthu <yogesh.mohanmarimuthu@amd.com> Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26547>
generated by cgit v1.2.3 (git 2.39.1) at 2024-05-09 06:30:10 +0000