diff options
| author | Dave Airlie <airlied@redhat.com> | 2014-02-06 00:49:58 +0000 |
|---|---|---|
| committer | Dave Airlie <airlied@redhat.com> | 2014-02-06 00:49:58 +0000 |
| commit | 1289080c4dd358b56301163ee18ee365df628b40 (patch) | |
| tree | e643d7137e4185acad0fc2f96bb935656199580e | |
| parent | 81144c049bc7c12e4edcdf28f91c3c024c6e8b2b (diff) | |
r600g: Add GL 3.3 support for 10.1 release
All patches on master below, except max samplers
which was removed on master.
Signed-off-by: Dave Airlie <airlied@redhat.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
commit 57c6bb18822ebf88a98b98714c846608ff3ba42b
Author: Dave Airlie <airlied@redhat.com>
Date: Thu Feb 6 00:48:57 2014 +0000
bump max samplers
commit 2e4bd244493bebd41edf725a2c3c4e793282a5bb
Author: Dave Airlie <airlied@redhat.com>
Date: Thu Jan 30 04:19:57 2014 +0000
r600g: add support for geom shaders to r600/r700 chipsets (v2)
This is my first attempt at enabling r600/r700 geometry shaders,
the basic tests pass on both my rv770 and my rv635,
It requires this kernel patch:
http://www.spinics.net/lists/dri-devel/msg52745.html
v2: address Alex comments.
Signed-off-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
commit 0ed4f769d77c4db2259befba5fc1707f1cb5cb98
Author: Dave Airlie <airlied@redhat.com>
Date: Wed Jan 29 21:48:09 2014 +0000
r600g: enable GLSL 3.30 on evergreen GPUs
This throws the switch to enable GL 3.3 and GLSL 330.
Signed-off-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
commit aeca8f21dd42b9ecd3932ef028fa8846036c1307
Author: Dave Airlie <airlied@redhat.com>
Date: Tue Feb 4 10:48:42 2014 +1000
r600g: properly propogate clip dist write value
This moves the value from the GS shader to the copy shader so the registers
are setup correctly.
fixes tests/spec/glsl-1.50/execution/geometry/clip-distance-out-values.shader_test
Signed-off-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
commit e1bc410fe670bb17078a55876f1700a504127fef
Author: Dave Airlie <airlied@redhat.com>
Date: Mon Feb 3 15:31:26 2014 +1000
r600g: calculate a better value for array_size (v2)
attempt to calculate a better value for array size to avoid breaking apps.
v2: use 0xfff like streamout, suggested by Grigori
Signed-off-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
commit 6f2f117dec51eb51c1b09e86e829e176a98e3bfc
Author: Dave Airlie <airlied@redhat.com>
Date: Fri Jan 31 03:35:51 2014 +0000
r600g: fix CAYMAN geometry shader support
cayman has a different end of program bit, so do that properly.
fixes hangs with geom shader tests on cayman.
Signed-off-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
commit 305ea22fd517f83406aba3e3930d710fd42a3049
Author: Dave Airlie <airlied@redhat.com>
Date: Wed Jan 29 00:17:15 2014 +0000
r600g: fix up shader out misc stuff for copy shader
set the correct values so the misc out register is setup correctly
for the copy shader.
This also updates the state for the gs copy shader so the hw
gets programmed correctly.
Signed-off-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
commit 53630e14c8791a84798a03d74653bf46bd013fc7
Author: Dave Airlie <airlied@redhat.com>
Date: Tue Jan 28 23:15:29 2014 +0000
r600g: port the layered surface rendering patch from radeonsi
This just makes r600 and evergreen do what the radeonsi codepaths do
for layered rendering. This makes the 2d amd_vertex_shader_layer test
pass on evergreen.
Signed-off-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
commit aa4cd3b9bed1ea23468fba4aa5c428153e8cddc1
Author: Dave Airlie <airlied@redhat.com>
Date: Tue Jan 28 13:04:00 2014 +1000
r600g: initial VS output layer support
This just adds support for emitting the proper value in the VS out misc.
Signed-off-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
commit 75a93f2e1e0f4d6015cdf63570ec4d3d12478b8d
Author: Dave Airlie <airlied@redhat.com>
Date: Tue Jan 28 12:06:49 2014 +1000
r600g: setup const texture buffers for geom shaders
This just enables the workarounds we have for vertex/pixel shaders
for geom shaders as well.
Signed-off-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
commit 88697a860635aae54e56dce2d6a839a06dea0c5a
Author: Dave Airlie <airlied@redhat.com>
Date: Fri Jan 24 17:14:26 2014 +1000
r600g: calculate correct cut value
This selects the cut value depending on the shader selected.
Signed-off-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
commit dfb88bef3e13112a838773e700c35052774f8a63
Author: Dave Airlie <airlied@redhat.com>
Date: Fri Jan 24 14:46:37 2014 +1000
r600g: fix dynamic_input_array_index.shader_test
This follows what fglrx does, it unpacks the input we are
going to indirect into a bunch of registers and indirects
inside them.
Signed-off-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
commit a3c6373f8cf3aab750399654a4b77150ec30bce9
Author: Dave Airlie <airlied@redhat.com>
Date: Fri Jan 24 13:39:36 2014 +1000
r600g: add support for indirect geom ring writes
We need to be able to write to the ring using a base register
for when we emit vertices in a loop, in theory the SB compiler
could collapse these indirect writes to direct writes if the
register value is constant and known, but that is outside my
pay grade.
Signed-off-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
commit dbc6a13adf935b118eaa6b396593f50d7b7e16e6
Author: Dave Airlie <airlied@redhat.com>
Date: Tue Dec 24 05:59:19 2013 +0000
r600g: write proper output prim type
Vadim's code derived it from the info.mode, but it needs
to be takes from the geometry shader output primitive.
Signed-off-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
commit f7f51b0b775f652967e2b972cf7c183482a771be
Author: Dave Airlie <airlied@redhat.com>
Date: Tue Dec 24 05:30:37 2013 +0000
r600g: enable instance cnt register with new enough kernel
The instance cnt register was missing for a few kernels,
with a new enough kernel we can output it.
Signed-off-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
commit 9e6ce37f66372018ec5398f74c3b43ff5f5bf309
Author: Dave Airlie <airlied@redhat.com>
Date: Mon Dec 23 01:30:03 2013 +0000
r600g: add primitive input support for gs
only enable prim id if gs uses it
Signed-off-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
commit fa932dfc7df3cf9ff63d08fb0e1db2119fc2ac93
Author: Dave Airlie <airlied@redhat.com>
Date: Thu Dec 19 05:17:00 2013 +0000
r600g: emit streamout from dma copy shader
This enables streamout with GS in the mix, from the
VS dma shader.
Signed-off-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
commit 205defb542ac185b7f46508fd51a4077a4702107
Author: Dave Airlie <airlied@redhat.com>
Date: Wed Dec 18 15:55:07 2013 +1000
r600g/gs: fix cases where number of gs inputs != number of gs outputs
this fixes a bunch of the geom shader built-in tests
Signed-off-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
commit d9e7ab40bc45644194c86f842599c76d0675243c
Author: Dave Airlie <airlied@redhat.com>
Date: Tue Jan 28 10:21:03 2014 +1000
r600g: increase array base for exported parameters
Trivial fix to Vadim's code.
Signed-off-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
commit 82d67fbd3b96b6b2cc0124a19b6f31b7912ec152
Author: Dave Airlie <airlied@redhat.com>
Date: Fri Jan 24 16:41:32 2014 +1000
r600g: initialise the geom shader loop registers.
As we do for vertex and pixel shaders.
Signed-off-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
commit 78be55d98d290d708bd1b3df3ef6cd5fa89865c7
Author: Dave Airlie <airlied@redhat.com>
Date: Sat Nov 30 06:26:13 2013 +0000
r600g: emit NOPs at end of shaders in more cases
If the shader has no CF clauses at all emit an nop
If the last instruction is an ENDLOOP add a NOP for the LOOP to go to
if the last instruction is CALL_FS add a NOP
These fix a bunch of hangs in the geometry shader tests.
Signed-off-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
commit 634b2498dc73efa3cca5a6fc3ed35c5bea6bb2e9
Author: Dave Airlie <airlied@redhat.com>
Date: Thu Nov 28 23:38:35 2013 +0000
r600g: don't enable SB for geom shaders
SB needs fixes for three GS instructions it seems to raise
them outside loops etc despite my best efforts.
Signed-off-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
commit 5b61dd0e917e54625ac227b8b1c2c82955f51ab1
Author: Dave Airlie <airlied@redhat.com>
Date: Tue Dec 24 04:56:25 2013 +0000
r600g/sb: add MEM_RING support
Although we don't use SB on geom shaders, the VS copy shader will use it
so we might as well implement MEM_RING support in sb.
Signed-off-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
commit 0247375aec4681c154ae4d14b8cd637e7a9e0e3e
Author: Dave Airlie <airlied@redhat.com>
Date: Wed Jan 29 04:08:43 2014 +0000
r600g: don't fail if we can't map VS->GS ring entries
This can happen in normal operation, so don't report an error on it,
just continue.
Signed-off-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
commit 2c986600fac6cb5692e9e377cb04f9f50389172c
Author: Vadim Girlin <vadimgirlin@gmail.com>
Date: Fri Aug 2 06:38:23 2013 +0400
r600g: initial support for geometry shaders on evergreen (v2)
This is Vadim's initial work with a few regression fixes squashed in.
v2: (airlied)
fix regression in glsl-max-varyings - need to use vs and ps_dirty
fix regression in shader exports from rebasing.
whitespace fixing.
v2.1: squash fix assert
Signed-off-by: Vadim Girlin <vadimgirlin@gmail.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
commit ce23c43e2b611f30964afe4d1c02c4d0361ba430
Author: Vadim Girlin <vadimgirlin@gmail.com>
Date: Fri Aug 2 06:32:32 2013 +0400
r600g: add hw register definitions for GS block setup
Signed-off-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
commit b0ec79c28d6373930ca0dc19168dd504204456b5
Author: Vadim Girlin <vadimgirlin@gmail.com>
Date: Wed Jul 31 23:09:39 2013 +0400
r600g: defer shader variant selection and depending state updates
[airlied: fix dropped streamout line - fix for master]
Signed-off-by: Vadim Girlin <vadimgirlin@gmail.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
commit e41cbfb4d15d519f9301699f39d7dd0153f2edf4
Author: Dave Airlie <airlied@redhat.com>
Date: Mon Jan 13 10:19:00 2014 +1000
r600g/bc: add support for indexed memory writes.
It looks like we need these for geom shaders in the future.
Signed-off-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
commit 46efb1648e883b2cb231cca38c1540e7e9ec1ecc
Author: Vadim Girlin <vadimgirlin@gmail.com>
Date: Wed Jul 31 20:02:22 2013 +0400
r600g: move barrier and end_of_program bits from output to cf struct (v2)
v2: fix regression on r600 NOP instructions.
Signed-off-by: Vadim Girlin <vadimgirlin@gmail.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
commit 42802d5d8d145f07cf3fca1bb6e8ab0cd1fd5c85
Author: Dave Airlie <airlied@redhat.com>
Date: Wed Jan 29 01:33:14 2014 +0000
r600g: split streamout emit code into a separate function
For geometry shaders we need to call this code from a second place.
Just move it out for now to keep future patches cleaner.
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
20 files changed, 1729 insertions, 484 deletions
diff --git a/src/gallium/drivers/r600/eg_asm.c b/src/gallium/drivers/r600/eg_asm.c index fffc436e823..acb30409428 100644 --- a/src/gallium/drivers/r600/eg_asm.c +++ b/src/gallium/drivers/r600/eg_asm.c @@ -79,45 +79,49 @@ int eg_bytecode_cf_build(struct r600_bytecode *bc, struct r600_bytecode_cf *cf) bc->bytecode[id++] = S_SQ_CF_ALLOC_EXPORT_WORD0_RW_GPR(cf->output.gpr) | S_SQ_CF_ALLOC_EXPORT_WORD0_ELEM_SIZE(cf->output.elem_size) | S_SQ_CF_ALLOC_EXPORT_WORD0_ARRAY_BASE(cf->output.array_base) | - S_SQ_CF_ALLOC_EXPORT_WORD0_TYPE(cf->output.type); + S_SQ_CF_ALLOC_EXPORT_WORD0_TYPE(cf->output.type) | + S_SQ_CF_ALLOC_EXPORT_WORD0_INDEX_GPR(cf->output.index_gpr); bc->bytecode[id] = S_SQ_CF_ALLOC_EXPORT_WORD1_BURST_COUNT(cf->output.burst_count - 1) | S_SQ_CF_ALLOC_EXPORT_WORD1_SWIZ_SEL_X(cf->output.swizzle_x) | S_SQ_CF_ALLOC_EXPORT_WORD1_SWIZ_SEL_Y(cf->output.swizzle_y) | S_SQ_CF_ALLOC_EXPORT_WORD1_SWIZ_SEL_Z(cf->output.swizzle_z) | S_SQ_CF_ALLOC_EXPORT_WORD1_SWIZ_SEL_W(cf->output.swizzle_w) | - S_SQ_CF_ALLOC_EXPORT_WORD1_BARRIER(cf->output.barrier) | + S_SQ_CF_ALLOC_EXPORT_WORD1_BARRIER(cf->barrier) | S_SQ_CF_ALLOC_EXPORT_WORD1_CF_INST(opcode); if (bc->chip_class == EVERGREEN) /* no EOP on cayman */ - bc->bytecode[id] |= S_SQ_CF_ALLOC_EXPORT_WORD1_END_OF_PROGRAM(cf->output.end_of_program); + bc->bytecode[id] |= S_SQ_CF_ALLOC_EXPORT_WORD1_END_OF_PROGRAM(cf->end_of_program); id++; - } else if (cfop->flags & CF_STRM) { - /* MEM_STREAM instructions */ + } else if (cfop->flags & CF_MEM) { + /* MEM_STREAM, MEM_RING instructions */ bc->bytecode[id++] = S_SQ_CF_ALLOC_EXPORT_WORD0_RW_GPR(cf->output.gpr) | S_SQ_CF_ALLOC_EXPORT_WORD0_ELEM_SIZE(cf->output.elem_size) | S_SQ_CF_ALLOC_EXPORT_WORD0_ARRAY_BASE(cf->output.array_base) | - S_SQ_CF_ALLOC_EXPORT_WORD0_TYPE(cf->output.type); + S_SQ_CF_ALLOC_EXPORT_WORD0_TYPE(cf->output.type) | + S_SQ_CF_ALLOC_EXPORT_WORD0_INDEX_GPR(cf->output.index_gpr); bc->bytecode[id] = S_SQ_CF_ALLOC_EXPORT_WORD1_BURST_COUNT(cf->output.burst_count - 1) | - S_SQ_CF_ALLOC_EXPORT_WORD1_BARRIER(cf->output.barrier) | + S_SQ_CF_ALLOC_EXPORT_WORD1_BARRIER(cf->barrier) | S_SQ_CF_ALLOC_EXPORT_WORD1_CF_INST(opcode) | S_SQ_CF_ALLOC_EXPORT_WORD1_BUF_COMP_MASK(cf->output.comp_mask) | S_SQ_CF_ALLOC_EXPORT_WORD1_BUF_ARRAY_SIZE(cf->output.array_size); if (bc->chip_class == EVERGREEN) /* no EOP on cayman */ - bc->bytecode[id] |= S_SQ_CF_ALLOC_EXPORT_WORD1_END_OF_PROGRAM(cf->output.end_of_program); + bc->bytecode[id] |= S_SQ_CF_ALLOC_EXPORT_WORD1_END_OF_PROGRAM(cf->end_of_program); id++; } else { - /* branch, loop, call, return instructions */ + /* other instructions */ bc->bytecode[id++] = S_SQ_CF_WORD0_ADDR(cf->cf_addr >> 1); bc->bytecode[id++] = S_SQ_CF_WORD1_CF_INST(opcode)| S_SQ_CF_WORD1_BARRIER(1) | S_SQ_CF_WORD1_COND(cf->cond) | - S_SQ_CF_WORD1_POP_COUNT(cf->pop_count); + S_SQ_CF_WORD1_POP_COUNT(cf->pop_count) | + S_SQ_CF_WORD1_END_OF_PROGRAM(cf->end_of_program); } } return 0; } +#if 0 void eg_bytecode_export_read(struct r600_bytecode *bc, struct r600_bytecode_output *output, uint32_t word0, uint32_t word1) { @@ -138,3 +142,4 @@ void eg_bytecode_export_read(struct r600_bytecode *bc, output->array_size = G_SQ_CF_ALLOC_EXPORT_WORD1_BUF_ARRAY_SIZE(word1); output->comp_mask = G_SQ_CF_ALLOC_EXPORT_WORD1_BUF_COMP_MASK(word1); } +#endif diff --git a/src/gallium/drivers/r600/evergreen_state.c b/src/gallium/drivers/r600/evergreen_state.c index 77e3c95636d..c86e812182b 100644 --- a/src/gallium/drivers/r600/evergreen_state.c +++ b/src/gallium/drivers/r600/evergreen_state.c @@ -1407,7 +1407,7 @@ void evergreen_init_color_surface(struct r600_context *rctx, struct pipe_resource *pipe_tex = surf->base.texture; unsigned level = surf->base.u.tex.level; unsigned pitch, slice; - unsigned color_info, color_attrib, color_dim = 0; + unsigned color_info, color_attrib, color_dim = 0, color_view; unsigned format, swap, ntype, endian; uint64_t offset, base_offset; unsigned non_disp_tiling, macro_aspect, tile_split, bankh, bankw, fmask_bankh, nbanks; @@ -1416,10 +1416,15 @@ void evergreen_init_color_surface(struct r600_context *rctx, bool blend_clamp = 0, blend_bypass = 0; offset = rtex->surface.level[level].offset; - if (rtex->surface.level[level].mode < RADEON_SURF_MODE_1D) { + if (rtex->surface.level[level].mode == RADEON_SURF_MODE_LINEAR) { + assert(surf->base.u.tex.first_layer == surf->base.u.tex.last_layer); offset += rtex->surface.level[level].slice_size * surf->base.u.tex.first_layer; - } + color_view = 0; + } else + color_view = S_028C6C_SLICE_START(surf->base.u.tex.first_layer) | + S_028C6C_SLICE_MAX(surf->base.u.tex.last_layer); + pitch = (rtex->surface.level[level].nblk_x) / 8 - 1; slice = (rtex->surface.level[level].nblk_x * rtex->surface.level[level].nblk_y) / 64; if (slice) { @@ -1569,12 +1574,7 @@ void evergreen_init_color_surface(struct r600_context *rctx, surf->cb_color_info = color_info; surf->cb_color_pitch = S_028C64_PITCH_TILE_MAX(pitch); surf->cb_color_slice = S_028C68_SLICE_TILE_MAX(slice); - if (rtex->surface.level[level].mode < RADEON_SURF_MODE_1D) { - surf->cb_color_view = 0; - } else { - surf->cb_color_view = S_028C6C_SLICE_START(surf->base.u.tex.first_layer) | - S_028C6C_SLICE_MAX(surf->base.u.tex.last_layer); - } + surf->cb_color_view = color_view; surf->cb_color_attrib = color_attrib; if (rtex->fmask.size) { surf->cb_color_fmask = (base_offset + rtex->fmask.offset) >> 8; @@ -1829,7 +1829,6 @@ static void evergreen_set_framebuffer_state(struct pipe_context *ctx, rctx->db_misc_state.atom.dirty = true; } - evergreen_update_db_shader_control(rctx); /* Calculate the CS size. */ rctx->framebuffer.atom.num_dw = 4; /* SCISSOR */ @@ -2519,6 +2518,7 @@ static void evergreen_emit_constant_buffers(struct r600_context *rctx, struct r600_resource *rbuffer; uint64_t va; unsigned buffer_index = ffs(dirty_mask) - 1; + unsigned gs_ring_buffer = (buffer_index == R600_GS_RING_CONST_BUFFER); cb = &state->cb[buffer_index]; rbuffer = (struct r600_resource*)cb->buffer; @@ -2527,10 +2527,12 @@ static void evergreen_emit_constant_buffers(struct r600_context *rctx, va = r600_resource_va(&rctx->screen->b.b, &rbuffer->b.b); va += cb->buffer_offset; - r600_write_context_reg_flag(cs, reg_alu_constbuf_size + buffer_index * 4, - ALIGN_DIVUP(cb->buffer_size >> 4, 16), pkt_flags); - r600_write_context_reg_flag(cs, reg_alu_const_cache + buffer_index * 4, va >> 8, - pkt_flags); + if (!gs_ring_buffer) { + r600_write_context_reg_flag(cs, reg_alu_constbuf_size + buffer_index * 4, + ALIGN_DIVUP(cb->buffer_size >> 4, 16), pkt_flags); + r600_write_context_reg_flag(cs, reg_alu_const_cache + buffer_index * 4, va >> 8, + pkt_flags); + } radeon_emit(cs, PKT3(PKT3_NOP, 0, 0) | pkt_flags); radeon_emit(cs, r600_context_bo_reloc(&rctx->b, &rctx->b.rings.gfx, rbuffer, RADEON_USAGE_READ)); @@ -2540,10 +2542,12 @@ static void evergreen_emit_constant_buffers(struct r600_context *rctx, radeon_emit(cs, va); /* RESOURCEi_WORD0 */ radeon_emit(cs, rbuffer->buf->size - cb->buffer_offset - 1); /* RESOURCEi_WORD1 */ radeon_emit(cs, /* RESOURCEi_WORD2 */ - S_030008_ENDIAN_SWAP(r600_endian_swap(32)) | - S_030008_STRIDE(16) | - S_030008_BASE_ADDRESS_HI(va >> 32UL)); + S_030008_ENDIAN_SWAP(gs_ring_buffer ? ENDIAN_NONE : r600_endian_swap(32)) | + S_030008_STRIDE(gs_ring_buffer ? 4 : 16) | + S_030008_BASE_ADDRESS_HI(va >> 32UL) | + S_030008_DATA_FORMAT(FMT_32_32_32_32_FLOAT)); radeon_emit(cs, /* RESOURCEi_WORD3 */ + S_03000C_UNCACHED(gs_ring_buffer ? 1 : 0) | S_03000C_DST_SEL_X(V_03000C_SQ_SEL_X) | S_03000C_DST_SEL_Y(V_03000C_SQ_SEL_Y) | S_03000C_DST_SEL_Z(V_03000C_SQ_SEL_Z) | @@ -2551,7 +2555,8 @@ static void evergreen_emit_constant_buffers(struct r600_context *rctx, radeon_emit(cs, 0); /* RESOURCEi_WORD4 */ radeon_emit(cs, 0); /* RESOURCEi_WORD5 */ radeon_emit(cs, 0); /* RESOURCEi_WORD6 */ - radeon_emit(cs, 0xc0000000); /* RESOURCEi_WORD7 */ + radeon_emit(cs, /* RESOURCEi_WORD7 */ + S_03001C_TYPE(V_03001C_SQ_TEX_VTX_VALID_BUFFER)); radeon_emit(cs, PKT3(PKT3_NOP, 0, 0) | pkt_flags); radeon_emit(cs, r600_context_bo_reloc(&rctx->b, &rctx->b.rings.gfx, rbuffer, RADEON_USAGE_READ)); @@ -2715,6 +2720,77 @@ static void evergreen_emit_vertex_fetch_shader(struct r600_context *rctx, struct radeon_emit(cs, r600_context_bo_reloc(&rctx->b, &rctx->b.rings.gfx, shader->buffer, RADEON_USAGE_READ)); } +static void evergreen_emit_shader_stages(struct r600_context *rctx, struct r600_atom *a) +{ + struct radeon_winsys_cs *cs = rctx->b.rings.gfx.cs; + struct r600_shader_stages_state *state = (struct r600_shader_stages_state*)a; + + uint32_t v = 0, v2 = 0, primid = 0; + + if (state->geom_enable) { + uint32_t cut_val; + + if (rctx->gs_shader->current->shader.gs_max_out_vertices <= 128) + cut_val = V_028A40_GS_CUT_128; + else if (rctx->gs_shader->current->shader.gs_max_out_vertices <= 256) + cut_val = V_028A40_GS_CUT_256; + else if (rctx->gs_shader->current->shader.gs_max_out_vertices <= 512) + cut_val = V_028A40_GS_CUT_512; + else + cut_val = V_028A40_GS_CUT_1024; + v = S_028B54_ES_EN(V_028B54_ES_STAGE_REAL) | + S_028B54_GS_EN(1) | + S_028B54_VS_EN(V_028B54_VS_STAGE_COPY_SHADER); + + v2 = S_028A40_MODE(V_028A40_GS_SCENARIO_G) | + S_028A40_CUT_MODE(cut_val); + + if (rctx->gs_shader->current->shader.gs_prim_id_input) + primid = 1; + } + + r600_write_context_reg(cs, R_028B54_VGT_SHADER_STAGES_EN, v); + r600_write_context_reg(cs, R_028A40_VGT_GS_MODE, v2); + r600_write_context_reg(cs, R_028A84_VGT_PRIMITIVEID_EN, primid); +} + +static void evergreen_emit_gs_rings(struct r600_context *rctx, struct r600_atom *a) +{ + struct pipe_screen *screen = rctx->b.b.screen; + struct radeon_winsys_cs *cs = rctx->b.rings.gfx.cs; + struct r600_gs_rings_state *state = (struct r600_gs_rings_state*)a; + struct r600_resource *rbuffer; + + r600_write_config_reg(cs, R_008040_WAIT_UNTIL, S_008040_WAIT_3D_IDLE(1)); + radeon_emit(cs, PKT3(PKT3_EVENT_WRITE, 0, 0)); + radeon_emit(cs, EVENT_TYPE(EVENT_TYPE_VGT_FLUSH)); + + if (state->enable) { + rbuffer =(struct r600_resource*)state->esgs_ring.buffer; + r600_write_config_reg(cs, R_008C40_SQ_ESGS_RING_BASE, + (r600_resource_va(screen, &rbuffer->b.b)) >> 8); + radeon_emit(cs, PKT3(PKT3_NOP, 0, 0)); + radeon_emit(cs, r600_context_bo_reloc(&rctx->b, &rctx->b.rings.gfx, rbuffer, RADEON_USAGE_READWRITE)); + r600_write_config_reg(cs, R_008C44_SQ_ESGS_RING_SIZE, + state->esgs_ring.buffer_size >> 8); + + rbuffer =(struct r600_resource*)state->gsvs_ring.buffer; + r600_write_config_reg(cs, R_008C48_SQ_GSVS_RING_BASE, + (r600_resource_va(screen, &rbuffer->b.b)) >> 8); + radeon_emit(cs, PKT3(PKT3_NOP, 0, 0)); + radeon_emit(cs, r600_context_bo_reloc(&rctx->b, &rctx->b.rings.gfx, rbuffer, RADEON_USAGE_READWRITE)); + r600_write_config_reg(cs, R_008C4C_SQ_GSVS_RING_SIZE, + state->gsvs_ring.buffer_size >> 8); + } else { + r600_write_config_reg(cs, R_008C44_SQ_ESGS_RING_SIZE, 0); + r600_write_config_reg(cs, R_008C4C_SQ_GSVS_RING_SIZE, 0); + } + + r600_write_config_reg(cs, R_008040_WAIT_UNTIL, S_008040_WAIT_3D_IDLE(1)); + radeon_emit(cs, PKT3(PKT3_EVENT_WRITE, 0, 0)); + radeon_emit(cs, EVENT_TYPE(EVENT_TYPE_VGT_FLUSH)); +} + void cayman_init_common_regs(struct r600_command_buffer *cb, enum chip_class ctx_chip_class, enum radeon_family ctx_family, @@ -2905,6 +2981,7 @@ static void cayman_init_atom_start_cs(struct r600_context *rctx) eg_store_loop_const(cb, R_03A200_SQ_LOOP_CONST_0, 0x01000FFF); eg_store_loop_const(cb, R_03A200_SQ_LOOP_CONST_0 + (32 * 4), 0x01000FFF); + eg_store_loop_const(cb, R_03A200_SQ_LOOP_CONST_0 + (64 * 4), 0x01000FFF); } void evergreen_init_common_regs(struct r600_command_buffer *cb, @@ -3363,6 +3440,7 @@ void evergreen_init_atom_start_cs(struct r600_context *rctx) eg_store_loop_const(cb, R_03A200_SQ_LOOP_CONST_0, 0x01000FFF); eg_store_loop_const(cb, R_03A200_SQ_LOOP_CONST_0 + (32 * 4), 0x01000FFF); + eg_store_loop_const(cb, R_03A200_SQ_LOOP_CONST_0 + (64 * 4), 0x01000FFF); } void evergreen_update_ps_state(struct pipe_context *ctx, struct r600_pipe_shader *shader) @@ -3510,6 +3588,102 @@ void evergreen_update_ps_state(struct pipe_context *ctx, struct r600_pipe_shader shader->flatshade = rctx->rasterizer->flatshade; } +void evergreen_update_es_state(struct pipe_context *ctx, struct r600_pipe_shader *shader) +{ + struct r600_command_buffer *cb = &shader->command_buffer; + struct r600_shader *rshader = &shader->shader; + + r600_init_command_buffer(cb, 32); + + r600_store_context_reg(cb, R_028890_SQ_PGM_RESOURCES_ES, + S_028890_NUM_GPRS(rshader->bc.ngpr) | + S_028890_STACK_SIZE(rshader->bc.nstack)); + r600_store_context_reg(cb, R_02888C_SQ_PGM_START_ES, + r600_resource_va(ctx->screen, (void *)shader->bo) >> 8); + /* After that, the NOP relocation packet must be emitted (shader->bo, RADEON_USAGE_READ). */ +} + +static unsigned r600_conv_prim_to_gs_out(unsigned mode) +{ + static const int prim_conv[] = { + V_028A6C_OUTPRIM_TYPE_POINTLIST, + V_028A6C_OUTPRIM_TYPE_LINESTRIP, + V_028A6C_OUTPRIM_TYPE_LINESTRIP, + V_028A6C_OUTPRIM_TYPE_LINESTRIP, + V_028A6C_OUTPRIM_TYPE_TRISTRIP, + V_028A6C_OUTPRIM_TYPE_TRISTRIP, + V_028A6C_OUTPRIM_TYPE_TRISTRIP, + V_028A6C_OUTPRIM_TYPE_TRISTRIP, + V_028A6C_OUTPRIM_TYPE_TRISTRIP, + V_028A6C_OUTPRIM_TYPE_TRISTRIP, + V_028A6C_OUTPRIM_TYPE_LINESTRIP, + V_028A6C_OUTPRIM_TYPE_LINESTRIP, + V_028A6C_OUTPRIM_TYPE_TRISTRIP, + V_028A6C_OUTPRIM_TYPE_TRISTRIP, + V_028A6C_OUTPRIM_TYPE_TRISTRIP + }; + assert(mode < Elements(prim_conv)); + + return prim_conv[mode]; +} + +void evergreen_update_gs_state(struct pipe_context *ctx, struct r600_pipe_shader *shader) +{ + struct r600_context *rctx = (struct r600_context *)ctx; + struct r600_command_buffer *cb = &shader->command_buffer; + struct r600_shader *rshader = &shader->shader; + struct r600_shader *cp_shader = &shader->gs_copy_shader->shader; + unsigned gsvs_itemsize = + (cp_shader->ring_item_size * rshader->gs_max_out_vertices) >> 2; + + r600_init_command_buffer(cb, 64); + + /* VGT_GS_MODE is written by evergreen_emit_shader_stages */ + + r600_store_context_reg(cb, R_028AB8_VGT_VTX_CNT_EN, 1); + + r600_store_context_reg(cb, R_028B38_VGT_GS_MAX_VERT_OUT, + S_028B38_MAX_VERT_OUT(rshader->gs_max_out_vertices)); + r600_store_context_reg(cb, R_028A6C_VGT_GS_OUT_PRIM_TYPE, + r600_conv_prim_to_gs_out(rshader->gs_output_prim)); + + if (rctx->screen->b.info.drm_minor >= 35) { + r600_store_context_reg(cb, R_028B90_VGT_GS_INSTANCE_CNT, + S_028B90_CNT(0) | + S_028B90_ENABLE(0)); + } + r600_store_context_reg_seq(cb, R_02891C_SQ_GS_VERT_ITEMSIZE, 4); + r600_store_value(cb, cp_shader->ring_item_size >> 2); + r600_store_value(cb, 0); + r600_store_value(cb, 0); + r600_store_value(cb, 0); + + r600_store_context_reg(cb, R_028900_SQ_ESGS_RING_ITEMSIZE, + (rshader->ring_item_size) >> 2); + + r600_store_context_reg(cb, R_028904_SQ_GSVS_RING_ITEMSIZE, + gsvs_itemsize); + + r600_store_context_reg_seq(cb, R_02892C_SQ_GSVS_RING_OFFSET_1, 3); + r600_store_value(cb, gsvs_itemsize); + r600_store_value(cb, gsvs_itemsize); + r600_store_value(cb, gsvs_itemsize); + + /* FIXME calculate these values somehow ??? */ + r600_store_context_reg_seq(cb, R_028A54_GS_PER_ES, 3); + r600_store_value(cb, 0x80); /* GS_PER_ES */ + r600_store_value(cb, 0x100); /* ES_PER_GS */ + r600_store_value(cb, 0x2); /* GS_PER_VS */ + + r600_store_context_reg(cb, R_028878_SQ_PGM_RESOURCES_GS, + S_028878_NUM_GPRS(rshader->bc.ngpr) | + S_028878_STACK_SIZE(rshader->bc.nstack)); + r600_store_context_reg(cb, R_028874_SQ_PGM_START_GS, + r600_resource_va(ctx->screen, (void *)shader->bo) >> 8); + /* After that, the NOP relocation packet must be emitted (shader->bo, RADEON_USAGE_READ). */ +} + + void evergreen_update_vs_state(struct pipe_context *ctx, struct r600_pipe_shader *shader) { struct r600_command_buffer *cb = &shader->command_buffer; @@ -3552,7 +3726,8 @@ void evergreen_update_vs_state(struct pipe_context *ctx, struct r600_pipe_shader S_02881C_VS_OUT_CCDIST0_VEC_ENA((rshader->clip_dist_write & 0x0F) != 0) | S_02881C_VS_OUT_CCDIST1_VEC_ENA((rshader->clip_dist_write & 0xF0) != 0) | S_02881C_VS_OUT_MISC_VEC_ENA(rshader->vs_out_misc_write) | - S_02881C_USE_VTX_POINT_SIZE(rshader->vs_out_point_size); + S_02881C_USE_VTX_POINT_SIZE(rshader->vs_out_point_size) | + S_02881C_USE_VTX_RENDER_TARGET_INDX(rshader->vs_out_layer); } void *evergreen_create_resolve_blend(struct r600_context *rctx) @@ -3919,6 +4094,10 @@ void evergreen_init_state_functions(struct r600_context *rctx) rctx->atoms[id++] = &rctx->b.streamout.begin_atom; r600_init_atom(rctx, &rctx->vertex_shader.atom, id++, r600_emit_shader, 23); r600_init_atom(rctx, &rctx->pixel_shader.atom, id++, r600_emit_shader, 0); + r600_init_atom(rctx, &rctx->geometry_shader.atom, id++, r600_emit_shader, 0); + r600_init_atom(rctx, &rctx->export_shader.atom, id++, r600_emit_shader, 0); + r600_init_atom(rctx, &rctx->shader_stages.atom, id++, evergreen_emit_shader_stages, 6); + r600_init_atom(rctx, &rctx->gs_rings.atom, id++, evergreen_emit_gs_rings, 26); rctx->b.b.create_blend_state = evergreen_create_blend_state; rctx->b.b.create_depth_stencil_alpha_state = evergreen_create_dsa_state; diff --git a/src/gallium/drivers/r600/evergreend.h b/src/gallium/drivers/r600/evergreend.h index 9ba3db7847c..986fd16f1ba 100644 --- a/src/gallium/drivers/r600/evergreend.h +++ b/src/gallium/drivers/r600/evergreend.h @@ -48,6 +48,7 @@ #define EVENT_TYPE_ZPASS_DONE 0x15 #define EVENT_TYPE_CACHE_FLUSH_AND_INV_EVENT 0x16 #define EVENT_TYPE_SO_VGTSTREAMOUT_FLUSH 0x1f +#define EVENT_TYPE_VGT_FLUSH 0x24 #define EVENT_TYPE_FLUSH_AND_INV_DB_META 0x2c #define EVENT_TYPE(x) ((x) << 0) @@ -274,6 +275,11 @@ #define G_008E2C_NUM_LS_LDS(x) (((x) >> 16) & 0xFFFF) #define C_008E2C_NUM_LS_LDS(x) 0xFFFF0000 +#define R_008C40_SQ_ESGS_RING_BASE 0x00008C40 +#define R_008C44_SQ_ESGS_RING_SIZE 0x00008C44 +#define R_008C48_SQ_GSVS_RING_BASE 0x00008C48 +#define R_008C4C_SQ_GSVS_RING_SIZE 0x00008C4C + #define R_008CF0_SQ_MS_FIFO_SIZES 0x00008CF0 #define S_008CF0_CACHE_FIFO_SIZE(x) (((x) & 0xFF) << 0) #define G_008CF0_CACHE_FIFO_SIZE(x) (((x) >> 0) & 0xFF) @@ -821,12 +827,22 @@ #define S_028A40_MODE(x) (((x) & 0x3) << 0) #define G_028A40_MODE(x) (((x) >> 0) & 0x3) #define C_028A40_MODE 0xFFFFFFFC +#define V_028A40_GS_OFF 0 +#define V_028A40_GS_SCENARIO_A 1 +#define V_028A40_GS_SCENARIO_B 2 +#define V_028A40_GS_SCENARIO_G 3 +#define V_028A40_GS_SCENARIO_C 4 +#define V_028A40_SPRITE_EN 5 #define S_028A40_ES_PASSTHRU(x) (((x) & 0x1) << 2) #define G_028A40_ES_PASSTHRU(x) (((x) >> 2) & 0x1) #define C_028A40_ES_PASSTHRU 0xFFFFFFFB #define S_028A40_CUT_MODE(x) (((x) & 0x3) << 3) #define G_028A40_CUT_MODE(x) (((x) >> 3) & 0x3) #define C_028A40_CUT_MODE 0xFFFFFFE7 +#define V_028A40_GS_CUT_1024 0 +#define V_028A40_GS_CUT_512 1 +#define V_028A40_GS_CUT_256 2 +#define V_028A40_GS_CUT_128 3 #define S_028A40_COMPUTE_MODE(x) (x << 14) #define S_028A40_PARTIAL_THD_AT_EOI(x) (x << 17) #define R_028A6C_VGT_GS_OUT_PRIM_TYPE 0x028A6C @@ -1201,6 +1217,7 @@ #define C_030008_ENDIAN_SWAP 0x3FFFFFFF #define R_03000C_SQ_VTX_CONSTANT_WORD3_0 0x03000C +#define S_03000C_UNCACHED(x) (((x) & 0x1) << 2) #define S_03000C_DST_SEL_X(x) (((x) & 0x7) << 3) #define G_03000C_DST_SEL_X(x) (((x) >> 3) & 0x7) #define V_03000C_SQ_SEL_X 0x00000000 @@ -1457,6 +1474,34 @@ #define G_028860_UNCACHED_FIRST_INST(x) (((x) >> 28) & 0x1) #define C_028860_UNCACHED_FIRST_INST 0xEFFFFFFF +#define R_028878_SQ_PGM_RESOURCES_GS 0x028878 +#define S_028878_NUM_GPRS(x) (((x) & 0xFF) << 0) +#define G_028878_NUM_GPRS(x) (((x) >> 0) & 0xFF) +#define C_028878_NUM_GPRS 0xFFFFFF00 +#define S_028878_STACK_SIZE(x) (((x) & 0xFF) << 8) +#define G_028878_STACK_SIZE(x) (((x) >> 8) & 0xFF) +#define C_028878_STACK_SIZE 0xFFFF00FF +#define S_028878_DX10_CLAMP(x) (((x) & 0x1) << 21) +#define G_028878_DX10_CLAMP(x) (((x) >> 21) & 0x1) +#define C_028878_DX10_CLAMP 0xFFDFFFFF +#define S_028878_UNCACHED_FIRST_INST(x) (((x) & 0x1) << 28) +#define G_028878_UNCACHED_FIRST_INST(x) (((x) >> 28) & 0x1) +#define C_028878_UNCACHED_FIRST_INST 0xEFFFFFFF + +#define R_028890_SQ_PGM_RESOURCES_ES 0x028890 +#define S_028890_NUM_GPRS(x) (((x) & 0xFF) << 0) +#define G_028890_NUM_GPRS(x) (((x) >> 0) & 0xFF) +#define C_028890_NUM_GPRS 0xFFFFFF00 +#define S_028890_STACK_SIZE(x) (((x) & 0xFF) << 8) +#define G_028890_STACK_SIZE(x) (((x) >> 8) & 0xFF) +#define C_028890_STACK_SIZE 0xFFFF00FF +#define S_028890_DX10_CLAMP(x) (((x) & 0x1) << 21) +#define G_028890_DX10_CLAMP(x) (((x) >> 21) & 0x1) +#define C_028890_DX10_CLAMP 0xFFDFFFFF +#define S_028890_UNCACHED_FIRST_INST(x) (((x) & 0x1) << 28) +#define G_028890_UNCACHED_FIRST_INST(x) (((x) >> 28) & 0x1) +#define C_028890_UNCACHED_FIRST_INST 0xEFFFFFFF + #define R_028864_SQ_PGM_RESOURCES_2_VS 0x028864 #define S_028864_SINGLE_ROUND(x) (((x) & 0x3) << 0) #define G_028864_SINGLE_ROUND(x) (((x) >> 0) & 0x3) @@ -1880,6 +1925,8 @@ #define G_02884C_EXPORT_Z(x) (((x) >> 0) & 0x1) #define C_02884C_EXPORT_Z 0xFFFFFFFE #define R_02885C_SQ_PGM_START_VS 0x0002885C +#define R_028874_SQ_PGM_START_GS 0x00028874 +#define R_02888C_SQ_PGM_START_ES 0x0002888C #define R_0288A4_SQ_PGM_START_FS 0x000288A4 #define R_0288D0_SQ_PGM_START_LS 0x000288d0 #define R_0288A8_SQ_PGM_RESOURCES_FS 0x000288A8 @@ -1894,6 +1941,9 @@ #define R_028920_SQ_GS_VERT_ITEMSIZE_1 0x00028920 #define R_028924_SQ_GS_VERT_ITEMSIZE_2 0x00028924 #define R_028928_SQ_GS_VERT_ITEMSIZE_3 0x00028928 +#define R_02892C_SQ_GSVS_RING_OFFSET_1 0x0002892C +#define R_028930_SQ_GSVS_RING_OFFSET_2 0x00028930 +#define R_028934_SQ_GSVS_RING_OFFSET_3 0x00028934 #define R_028940_ALU_CONST_CACHE_PS_0 0x00028940 #define R_028944_ALU_CONST_CACHE_PS_1 0x00028944 #define R_028980_ALU_CONST_CACHE_VS_0 0x00028980 @@ -1928,6 +1978,15 @@ #define S_028A48_VPORT_SCISSOR_ENABLE(x) (((x) & 0x1) << 1) #define S_028A48_LINE_STIPPLE_ENABLE(x) (((x) & 0x1) << 2) #define R_028A4C_PA_SC_MODE_CNTL_1 0x00028A4C + +#define R_028A54_GS_PER_ES 0x00028A54 +#define R_028A58_ES_PER_GS 0x00028A58 +#define R_028A5C_GS_PER_VS 0x00028A5C + +#define R_028A84_VGT_PRIMITIVEID_EN 0x028A84 +#define S_028A84_PRIMITIVEID_EN(x) (((x) & 0x1) << 0) +#define G_028A84_PRIMITIVEID_EN(x) (((x) >> 0) & 0x1) +#define C_028A84_PRIMITIVEID_EN 0xFFFFFFFE #define R_028A94_VGT_MULTI_PRIM_IB_RESET_EN 0x00028A94 #define S_028A94_RESET_EN(x) (((x) & 0x1) << 0) #define G_028A94_RESET_EN(x) (((x) >> 0) & 0x1) @@ -1962,11 +2021,27 @@ #define R_028B28_VGT_STRMOUT_DRAW_OPAQUE_OFFSET 0x028B28 #define R_028B2C_VGT_STRMOUT_DRAW_OPAQUE_BUFFER_FILLED_SIZE 0x028B2C #define R_028B30_VGT_STRMOUT_DRAW_OPAQUE_VERTEX_STRIDE 0x028B30 +#define R_028B38_VGT_GS_MAX_VERT_OUT 0x028B38 +#define S_028B38_MAX_VERT_OUT(x) (((x) & 0x7FF) << 0) #define R_028B44_VGT_STRMOUT_BASE_OFFSET_HI_0 0x028B44 #define R_028B48_VGT_STRMOUT_BASE_OFFSET_HI_1 0x028B48 #define R_028B4C_VGT_STRMOUT_BASE_OFFSET_HI_2 0x028B4C #define R_028B50_VGT_STRMOUT_BASE_OFFSET_HI_3 0x028B50 #define R_028B54_VGT_SHADER_STAGES_EN 0x00028B54 +#define S_028B54_LS_EN(x) (((x) & 0x3) << 0) +#define V_028B54_LS_STAGE_OFF 0x00 +#define V_028B54_LS_STAGE_ON 0x01 +#define V_028B54_CS_STAGE_ON 0x02 +#define S_028B54_HS_EN(x) (((x) & 0x1) << 2) +#define S_028B54_ES_EN(x) (((x) & 0x3) << 3) +#define V_028B54_ES_STAGE_OFF 0x00 +#define V_028B54_ES_STAGE_DS 0x01 +#define V_028B54_ES_STAGE_REAL 0x02 +#define S_028B54_GS_EN(x) (((x) & 0x1) << 5) +#define S_028B54_VS_EN(x) (((x) & 0x3) << 6) +#define V_028B54_VS_STAGE_REAL 0x00 +#define V_028B54_VS_STAGE_DS 0x01 +#define V_028B54_VS_STAGE_COPY_SHADER 0x02 #define R_028B70_DB_ALPHA_TO_MASK 0x00028B70 #define S_028B70_ALPHA_TO_MASK_ENABLE(x) (((x) & 0x1) << 0) #define S_028B70_ALPHA_TO_MASK_OFFSET0(x) (((x) & 0x3) << 8) @@ -1998,12 +2073,9 @@ #define S_028B8C_OFFSET(x) (((x) & 0xFFFFFFFF) << 0) #define G_028B8C_OFFSET(x) (((x) >> 0) & 0xFFFFFFFF) #define C_028B8C_OFFSET 0x00000000 -#define R_028B94_VGT_STRMOUT_CONFIG 0x028B94 -#define S_028B94_STREAMOUT_0_EN(x) (((x) & 0x1) << 0) -#define S_028B94_STREAMOUT_1_EN(x) (((x) & 0x1) << 1) -#define S_028B94_STREAMOUT_2_EN(x) (((x) & 0x1) << 2) -#define S_028B94_STREAMOUT_3_EN(x) (((x) & 0x1) << 3) -#define S_028B94_RAST_STREAM(x) (((x) & 0x07) << 4) +#define R_028B90_VGT_GS_INSTANCE_CNT 0x00028B90 +#define S_028B90_ENABLE(x) (((x) & 0x1) << 0) +#define S_028B90_CNT(x) (((x) & 0x7F) << 2) #define R_028B98_VGT_STRMOUT_BUFFER_CONFIG 0x028B98 #define S_028B98_STREAM_0_BUFFER_EN(x) (((x) & 0x0F) << 0) #define S_028B98_STREAM_1_BUFFER_EN(x) (((x) & 0x0F) << 4) diff --git a/src/gallium/drivers/r600/r600_asm.c b/src/gallium/drivers/r600/r600_asm.c index c5922a8fa5b..3afe7b2c019 100644 --- a/src/gallium/drivers/r600/r600_asm.c +++ b/src/gallium/drivers/r600/r600_asm.c @@ -193,7 +193,6 @@ int r600_bytecode_add_output(struct r600_bytecode *bc, if ((output->gpr + output->burst_count) == bc->cf_last->output.gpr && (output->array_base + output->burst_count) == bc->cf_last->output.array_base) { - bc->cf_last->output.end_of_program |= output->end_of_program; bc->cf_last->op = bc->cf_last->output.op = output->op; bc->cf_last->output.gpr = output->gpr; bc->cf_last->output.array_base = output->array_base; @@ -203,7 +202,6 @@ int r600_bytecode_add_output(struct r600_bytecode *bc, } else if (output->gpr == (bc->cf_last->output.gpr + bc->cf_last->output.burst_count) && output->array_base == (bc->cf_last->output.array_base + bc->cf_last->output.burst_count)) { - bc->cf_last->output.end_of_program |= output->end_of_program; bc->cf_last->op = bc->cf_last->output.op = output->op; bc->cf_last->output.burst_count += output->burst_count; return 0; @@ -215,6 +213,7 @@ int r600_bytecode_add_output(struct r600_bytecode *bc, return r; bc->cf_last->op = output->op; memcpy(&bc->cf_last->output, output, sizeof(struct r600_bytecode_output)); + bc->cf_last->barrier = 1; return 0; } @@ -1526,24 +1525,26 @@ static int r600_bytecode_cf_build(struct r600_bytecode *bc, struct r600_bytecode bc->bytecode[id++] = S_SQ_CF_ALLOC_EXPORT_WORD0_RW_GPR(cf->output.gpr) | S_SQ_CF_ALLOC_EXPORT_WORD0_ELEM_SIZE(cf->output.elem_size) | S_SQ_CF_ALLOC_EXPORT_WORD0_ARRAY_BASE(cf->output.array_base) | - S_SQ_CF_ALLOC_EXPORT_WORD0_TYPE(cf->output.type); + S_SQ_CF_ALLOC_EXPORT_WORD0_TYPE(cf->output.type) | + S_SQ_CF_ALLOC_EXPORT_WORD0_INDEX_GPR(cf->output.index_gpr); bc->bytecode[id++] = S_SQ_CF_ALLOC_EXPORT_WORD1_BURST_COUNT(cf->output.burst_count - 1) | S_SQ_CF_ALLOC_EXPORT_WORD1_SWIZ_SEL_X(cf->output.swizzle_x) | S_SQ_CF_ALLOC_EXPORT_WORD1_SWIZ_SEL_Y(cf->output.swizzle_y) | S_SQ_CF_ALLOC_EXPORT_WORD1_SWIZ_SEL_Z(cf->output.swizzle_z) | S_SQ_CF_ALLOC_EXPORT_WORD1_SWIZ_SEL_W(cf->output.swizzle_w) | - S_SQ_CF_ALLOC_EXPORT_WORD1_BARRIER(cf->output.barrier) | + S_SQ_CF_ALLOC_EXPORT_WORD1_BARRIER(cf->barrier) | S_SQ_CF_ALLOC_EXPORT_WORD1_CF_INST(opcode) | - S_SQ_CF_ALLOC_EXPORT_WORD1_END_OF_PROGRAM(cf->output.end_of_program); - } else if (cfop->flags & CF_STRM) { + S_SQ_CF_ALLOC_EXPORT_WORD1_END_OF_PROGRAM(cf->end_of_program); + } else if (cfop->flags & CF_MEM) { bc->bytecode[id++] = S_SQ_CF_ALLOC_EXPORT_WORD0_RW_GPR(cf->output.gpr) | S_SQ_CF_ALLOC_EXPORT_WORD0_ELEM_SIZE(cf->output.elem_size) | S_SQ_CF_ALLOC_EXPORT_WORD0_ARRAY_BASE(cf->output.array_base) | - S_SQ_CF_ALLOC_EXPORT_WORD0_TYPE(cf->output.type); + S_SQ_CF_ALLOC_EXPORT_WORD0_TYPE(cf->output.type) | + S_SQ_CF_ALLOC_EXPORT_WORD0_INDEX_GPR(cf->output.index_gpr); bc->bytecode[id++] = S_SQ_CF_ALLOC_EXPORT_WORD1_BURST_COUNT(cf->output.burst_count - 1) | - S_SQ_CF_ALLOC_EXPORT_WORD1_BARRIER(cf->output.barrier) | + S_SQ_CF_ALLOC_EXPORT_WORD1_BARRIER(cf->barrier) | S_SQ_CF_ALLOC_EXPORT_WORD1_CF_INST(opcode) | - S_SQ_CF_ALLOC_EXPORT_WORD1_END_OF_PROGRAM(cf->output.end_of_program) | + S_SQ_CF_ALLOC_EXPORT_WORD1_END_OF_PROGRAM(cf->end_of_program) | S_SQ_CF_ALLOC_EXPORT_WORD1_BUF_ARRAY_SIZE(cf->output.array_size) | S_SQ_CF_ALLOC_EXPORT_WORD1_BUF_COMP_MASK(cf->output.comp_mask); } else { @@ -1551,7 +1552,8 @@ static int r600_bytecode_cf_build(struct r600_bytecode *bc, struct r600_bytecode bc->bytecode[id++] = S_SQ_CF_WORD1_CF_INST(opcode) | S_SQ_CF_WORD1_BARRIER(1) | S_SQ_CF_WORD1_COND(cf->cond) | - S_SQ_CF_WORD1_POP_COUNT(cf->pop_count); + S_SQ_CF_WORD1_POP_COUNT(cf->pop_count) | + S_SQ_CF_WORD1_END_OF_PROGRAM(cf->end_of_program); } return 0; } @@ -1932,12 +1934,12 @@ void r600_bytecode_disasm(struct r600_bytecode *bc) print_indent(o, 67); fprintf(stderr, " ES:%X ", cf->output.elem_size); - if (!cf->output.barrier) + if (!cf->barrier) fprintf(stderr, "NO_BARRIER "); - if (cf->output.end_of_program) + if (cf->end_of_program) fprintf(stderr, "EOP "); fprintf(stderr, "\n"); - } else if (r600_isa_cf(cf->op)->flags & CF_STRM) { + } else if (r600_isa_cf(cf->op)->flags & CF_MEM) { int o = 0; const char *exp_type[] = {"WRITE", "WRITE_IND", "WRITE_ACK", "WRITE_IND_ACK"}; @@ -1963,14 +1965,17 @@ void r600_bytecode_disasm(struct r600_bytecode *bc) o += print_swizzle(7); } + if (cf->output.type == V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_WRITE_IND) + o += fprintf(stderr, " R%d", cf->output.index_gpr); + o += print_indent(o, 67); fprintf(stderr, " ES:%i ", cf->output.elem_size); if (cf->output.array_size != 0xFFF) fprintf(stderr, "AS:%i ", cf->output.array_size); - if (!cf->output.barrier) + if (!cf->barrier) fprintf(stderr, "NO_BARRIER "); - if (cf->output.end_of_program) + if (cf->end_of_program) fprintf(stderr, "EOP "); fprintf(stderr, "\n"); } else { @@ -2486,6 +2491,7 @@ void r600_bytecode_alu_read(struct r600_bytecode *bc, } } +#if 0 void r600_bytecode_export_read(struct r600_bytecode *bc, struct r600_bytecode_output *output, uint32_t word0, uint32_t word1) { @@ -2506,3 +2512,4 @@ void r600_bytecode_export_read(struct r600_bytecode *bc, output->array_size = G_SQ_CF_ALLOC_EXPORT_WORD1_BUF_ARRAY_SIZE(word1); output->comp_mask = G_SQ_CF_ALLOC_EXPORT_WORD1_BUF_COMP_MASK(word1); } +#endif diff --git a/src/gallium/drivers/r600/r600_asm.h b/src/gallium/drivers/r600/r600_asm.h index 82c6c8d1c7e..4fb0ef06956 100644 --- a/src/gallium/drivers/r600/r600_asm.h +++ b/src/gallium/drivers/r600/r600_asm.h @@ -115,7 +115,6 @@ struct r600_bytecode_output { unsigned array_size; unsigned comp_mask; unsigned type; - unsigned end_of_program; unsigned op; @@ -126,7 +125,7 @@ struct r600_bytecode_output { unsigned swizzle_z; unsigned swizzle_w; unsigned burst_count; - unsigned barrier; + unsigned index_gpr; }; struct r600_bytecode_kcache { @@ -148,6 +147,8 @@ struct r600_bytecode_cf { struct r600_bytecode_kcache kcache[4]; unsigned r6xx_uses_waterfall; unsigned eg_alu_extended; + unsigned barrier; + unsigned end_of_program; struct list_head alu; struct list_head tex; struct list_head vtx; diff --git a/src/gallium/drivers/r600/r600_blit.c b/src/gallium/drivers/r600/r600_blit.c index c2ae2f6b28e..6bb7cfe9b9e 100644 --- a/src/gallium/drivers/r600/r600_blit.c +++ b/src/gallium/drivers/r600/r600_blit.c @@ -59,6 +59,7 @@ static void r600_blitter_begin(struct pipe_context *ctx, enum r600_blitter_op op util_blitter_save_vertex_buffer_slot(rctx->blitter, rctx->vertex_buffer_state.vb); util_blitter_save_vertex_elements(rctx->blitter, rctx->vertex_fetch_shader.cso); util_blitter_save_vertex_shader(rctx->blitter, rctx->vs_shader); + util_blitter_save_geometry_shader(rctx->blitter, rctx->gs_shader); util_blitter_save_so_targets(rctx->blitter, rctx->b.streamout.num_targets, (struct pipe_stream_output_target**)rctx->b.streamout.targets); util_blitter_save_rasterizer(rctx->blitter, rctx->rasterizer_state.cso); diff --git a/src/gallium/drivers/r600/r600_hw_context.c b/src/gallium/drivers/r600/r600_hw_context.c index 9b0c558c29e..fc81e95accd 100644 --- a/src/gallium/drivers/r600/r600_hw_context.c +++ b/src/gallium/drivers/r600/r600_hw_context.c @@ -301,6 +301,12 @@ void r600_begin_new_cs(struct r600_context *ctx) ctx->config_state.atom.dirty = true; ctx->stencil_ref.atom.dirty = true; ctx->vertex_fetch_shader.atom.dirty = true; + ctx->export_shader.atom.dirty = true; + if (ctx->gs_shader) { + ctx->geometry_shader.atom.dirty = true; + ctx->shader_stages.atom.dirty = true; + ctx->gs_rings.atom.dirty = true; + } ctx->vertex_shader.atom.dirty = true; ctx->viewport.atom.dirty = true; diff --git a/src/gallium/drivers/r600/r600_pipe.c b/src/gallium/drivers/r600/r600_pipe.c index 07f6c1ce566..edf37ade14f 100644 --- a/src/gallium/drivers/r600/r600_pipe.c +++ b/src/gallium/drivers/r600/r600_pipe.c @@ -372,6 +372,11 @@ static int r600_get_param(struct pipe_screen* pscreen, enum pipe_cap param) return 1; case PIPE_CAP_GLSL_FEATURE_LEVEL: + if (family >= CHIP_CEDAR) + return 330; + /* pre-evergreen geom shaders need newer kernel */ + if (rscreen->b.info.drm_minor >= 37) + return 330; return 140; /* Supported except the original R600. */ @@ -383,6 +388,7 @@ static int r600_get_param(struct pipe_screen* pscreen, enum pipe_cap param) /* Supported on Evergreen. */ case PIPE_CAP_SEAMLESS_CUBE_MAP_PER_TEXTURE: case PIPE_CAP_CUBE_MAP_ARRAY: + case PIPE_CAP_TGSI_VS_LAYER: return family >= CHIP_CEDAR ? 1 : 0; /* Unsupported features. */ @@ -392,7 +398,6 @@ static int r600_get_param(struct pipe_screen* pscreen, enum pipe_cap param) case PIPE_CAP_FRAGMENT_COLOR_CLAMPED: case PIPE_CAP_VERTEX_COLOR_CLAMPED: case PIPE_CAP_USER_VERTEX_BUFFERS: - case PIPE_CAP_TGSI_VS_LAYER: return 0; /* Stream output. */ @@ -416,7 +421,7 @@ static int r600_get_param(struct pipe_screen* pscreen, enum pipe_cap param) return rscreen->b.info.drm_minor >= 9 ? (family >= CHIP_CEDAR ? 16384 : 8192) : 0; case PIPE_CAP_MAX_COMBINED_SAMPLERS: - return 32; + return 48; /* Render targets. */ case PIPE_CAP_MAX_RENDER_TARGETS: @@ -449,14 +454,20 @@ static int r600_get_param(struct pipe_screen* pscreen, enum pipe_cap param) static int r600_get_shader_param(struct pipe_screen* pscreen, unsigned shader, enum pipe_shader_cap param) { + struct r600_screen *rscreen = (struct r600_screen *)pscreen; + switch(shader) { case PIPE_SHADER_FRAGMENT: case PIPE_SHADER_VERTEX: - case PIPE_SHADER_COMPUTE: + case PIPE_SHADER_COMPUTE: break; case PIPE_SHADER_GEOMETRY: - /* XXX: support and enable geometry programs */ + if (rscreen->b.family >= CHIP_CEDAR) + break; + /* pre-evergreen geom shaders need newer kernel */ + if (rscreen->b.info.drm_minor >= 37) + break; return 0; default: /* XXX: support tessellation on Evergreen */ diff --git a/src/gallium/drivers/r600/r600_pipe.h b/src/gallium/drivers/r600/r600_pipe.h index 31e27f282e5..9f27a17cd53 100644 --- a/src/gallium/drivers/r600/r600_pipe.h +++ b/src/gallium/drivers/r600/r600_pipe.h @@ -38,7 +38,7 @@ #include "util/u_double_list.h" #include "util/u_transfer.h" -#define R600_NUM_ATOMS 41 +#define R600_NUM_ATOMS 42 /* the number of CS dwords for flushing and drawing */ #define R600_MAX_FLUSH_CS_DWORDS 16 @@ -46,13 +46,14 @@ #define R600_TRACE_CS_DWORDS 7 #define R600_MAX_USER_CONST_BUFFERS 13 -#define R600_MAX_DRIVER_CONST_BUFFERS 3 +#define R600_MAX_DRIVER_CONST_BUFFERS 4 #define R600_MAX_CONST_BUFFERS (R600_MAX_USER_CONST_BUFFERS + R600_MAX_DRIVER_CONST_BUFFERS) /* start driver buffers after user buffers */ #define R600_UCP_CONST_BUFFER (R600_MAX_USER_CONST_BUFFERS) #define R600_TXQ_CONST_BUFFER (R600_MAX_USER_CONST_BUFFERS + 1) #define R600_BUFFER_INFO_CONST_BUFFER (R600_MAX_USER_CONST_BUFFERS + 2) +#define R600_GS_RING_CONST_BUFFER (R600_MAX_USER_CONST_BUFFERS + 3) #define R600_MAX_CONST_BUFFER_SIZE 4096 @@ -159,6 +160,7 @@ struct r600_sample_mask { struct r600_config_state { struct r600_atom atom; unsigned sq_gpr_resource_mgmt_1; + unsigned sq_gpr_resource_mgmt_2; }; struct r600_stencil_ref @@ -179,6 +181,18 @@ struct r600_viewport_state { struct pipe_viewport_state state; }; +struct r600_shader_stages_state { + struct r600_atom atom; + unsigned geom_enable; +}; + +struct r600_gs_rings_state { + struct r600_atom atom; + unsigned enable; + struct pipe_constant_buffer esgs_ring; + struct pipe_constant_buffer gsvs_ring; +}; + /* This must start from 16. */ /* features */ #define DBG_NO_LLVM (1 << 17) @@ -353,7 +367,7 @@ struct r600_fetch_shader { struct r600_shader_state { struct r600_atom atom; - struct r600_pipe_shader_selector *shader; + struct r600_pipe_shader *shader; }; struct r600_context { @@ -415,7 +429,11 @@ struct r600_context { struct r600_cso_state vertex_fetch_shader; struct r600_shader_state vertex_shader; struct r600_shader_state pixel_shader; + struct r600_shader_state geometry_shader; + struct r600_shader_state export_shader; struct r600_cs_shader_state cs_shader_state; + struct r600_shader_stages_state shader_stages; + struct r600_gs_rings_state gs_rings; struct r600_constbuf_state constbuf_state[PIPE_SHADER_TYPES]; struct r600_textures_info samplers[PIPE_SHADER_TYPES]; /** Vertex buffers for fetch shaders */ @@ -427,6 +445,7 @@ struct r600_context { unsigned compute_cb_target_mask; struct r600_pipe_shader_selector *ps_shader; struct r600_pipe_shader_selector *vs_shader; + struct r600_pipe_shader_selector *gs_shader; struct r600_rasterizer_state *rasterizer; bool alpha_to_one; bool force_blend_disable; @@ -506,6 +525,8 @@ void cayman_init_common_regs(struct r600_command_buffer *cb, void evergreen_init_state_functions(struct r600_context *rctx); void evergreen_init_atom_start_cs(struct r600_context *rctx); void evergreen_update_ps_state(struct pipe_context *ctx, struct r600_pipe_shader *shader); +void evergreen_update_es_state(struct pipe_context *ctx, struct r600_pipe_shader *shader); +void evergreen_update_gs_state(struct pipe_context *ctx, struct r600_pipe_shader *shader); void evergreen_update_vs_state(struct pipe_context *ctx, struct r600_pipe_shader *shader); void *evergreen_create_db_flush_dsa(struct r600_context *rctx); void *evergreen_create_resolve_blend(struct r600_context *rctx); @@ -545,6 +566,8 @@ r600_create_sampler_view_custom(struct pipe_context *ctx, void r600_init_state_functions(struct r600_context *rctx); void r600_init_atom_start_cs(struct r600_context *rctx); void r600_update_ps_state(struct pipe_context *ctx, struct r600_pipe_shader *shader); +void r600_update_es_state(struct pipe_context *ctx, struct r600_pipe_shader *shader); +void r600_update_gs_state(struct pipe_context *ctx, struct r600_pipe_shader *shader); void r600_update_vs_state(struct pipe_context *ctx, struct r600_pipe_shader *shader); void *r600_create_db_flush_dsa(struct r600_context *rctx); void *r600_create_resolve_blend(struct r600_context *rctx); diff --git a/src/gallium/drivers/r600/r600_shader.c b/src/gallium/drivers/r600/r600_shader.c index 1ea4ae6c056..758abd76bf4 100644 --- a/src/gallium/drivers/r600/r600_shader.c +++ b/src/gallium/drivers/r600/r600_shader.c @@ -60,7 +60,7 @@ issued in the w slot as well. The compiler must issue the source argument to slots z, y, and x */ -static int r600_shader_from_tgsi(struct r600_screen *rscreen, +static int r600_shader_from_tgsi(struct r600_context *rctx, struct r600_pipe_shader *pipeshader, struct r600_shader_key key); @@ -104,17 +104,43 @@ static void r600_dump_streamout(struct pipe_stream_output_info *so) } } +static int store_shader(struct pipe_context *ctx, + struct r600_pipe_shader *shader) +{ + struct r600_context *rctx = (struct r600_context *)ctx; + uint32_t *ptr, i; + + if (shader->bo == NULL) { + shader->bo = (struct r600_resource*) + pipe_buffer_create(ctx->screen, PIPE_BIND_CUSTOM, PIPE_USAGE_IMMUTABLE, shader->shader.bc.ndw * 4); + if (shader->bo == NULL) { + return -ENOMEM; + } + ptr = r600_buffer_map_sync_with_rings(&rctx->b, shader->bo, PIPE_TRANSFER_WRITE); + if (R600_BIG_ENDIAN) { + for (i = 0; i < shader->shader.bc.ndw; ++i) { + ptr[i] = util_bswap32(shader->shader.bc.bytecode[i]); + } + } else { + memcpy(ptr, shader->shader.bc.bytecode, shader->shader.bc.ndw * sizeof(*ptr)); + } + rctx->b.ws->buffer_unmap(shader->bo->cs_buf); + } + + return 0; +} + int r600_pipe_shader_create(struct pipe_context *ctx, struct r600_pipe_shader *shader, struct r600_shader_key key) { struct r600_context *rctx = (struct r600_context *)ctx; struct r600_pipe_shader_selector *sel = shader->selector; - int r, i; - uint32_t *ptr; + int r; bool dump = r600_can_dump_shader(&rctx->screen->b, sel->tokens); unsigned use_sb = !(rctx->screen->b.debug_flags & DBG_NO_SB); unsigned sb_disasm = use_sb || (rctx->screen->b.debug_flags & DBG_SB_DISASM); + unsigned export_shader = key.vs_as_es; shader->shader.bc.isa = rctx->isa; @@ -126,12 +152,15 @@ int r600_pipe_shader_create(struct pipe_context *ctx, r600_dump_streamout(&sel->so); } } - r = r600_shader_from_tgsi(rctx->screen, shader, key); + r = r600_shader_from_tgsi(rctx, shader, key); if (r) { R600_ERR("translation from TGSI failed !\n"); return r; } + /* disable SB for geom shaders - it can't handle the CF_EMIT instructions */ + use_sb &= (shader->shader.processor_type != TGSI_PROCESSOR_GEOMETRY); + /* Check if the bytecode has already been built. When using the llvm * backend, r600_shader_from_tgsi() will take care of building the * bytecode. @@ -157,31 +186,45 @@ int r600_pipe_shader_create(struct pipe_context *ctx, } } - /* Store the shader in a buffer. */ - if (shader->bo == NULL) { - shader->bo = (struct r600_resource*) - pipe_buffer_create(ctx->screen, PIPE_BIND_CUSTOM, PIPE_USAGE_IMMUTABLE, shader->shader.bc.ndw * 4); - if (shader->bo == NULL) { - return -ENOMEM; - } - ptr = r600_buffer_map_sync_with_rings(&rctx->b, shader->bo, PIPE_TRANSFER_WRITE); - if (R600_BIG_ENDIAN) { - for (i = 0; i < shader->shader.bc.ndw; ++i) { - ptr[i] = util_bswap32(shader->shader.bc.bytecode[i]); - } - } else { - memcpy(ptr, shader->shader.bc.bytecode, shader->shader.bc.ndw * sizeof(*ptr)); + if (shader->gs_copy_shader) { + if (dump) { + // dump copy shader + r = r600_sb_bytecode_process(rctx, &shader->gs_copy_shader->shader.bc, + &shader->gs_copy_shader->shader, dump, 0); + if (r) + return r; } - rctx->b.ws->buffer_unmap(shader->bo->cs_buf); + + if ((r = store_shader(ctx, shader->gs_copy_shader))) + return r; } + /* Store the shader in a buffer. */ + if ((r = store_shader(ctx, shader))) + return r; + /* Build state. */ switch (shader->shader.processor_type) { + case TGSI_PROCESSOR_GEOMETRY: + if (rctx->b.chip_class >= EVERGREEN) { + evergreen_update_gs_state(ctx, shader); + evergreen_update_vs_state(ctx, shader->gs_copy_shader); + } else { + r600_update_gs_state(ctx, shader); + r600_update_vs_state(ctx, shader->gs_copy_shader); + } + break; case TGSI_PROCESSOR_VERTEX: if (rctx->b.chip_class >= EVERGREEN) { - evergreen_update_vs_state(ctx, shader); + if (export_shader) + evergreen_update_es_state(ctx, shader); + else + evergreen_update_vs_state(ctx, shader); } else { - r600_update_vs_state(ctx, shader); + if (export_shader) + r600_update_es_state(ctx, shader); + else + r600_update_vs_state(ctx, shader); } break; case TGSI_PROCESSOR_FRAGMENT: @@ -245,6 +288,11 @@ struct r600_shader_ctx { unsigned cv_output; int fragcoord_input; int native_integers; + int next_ring_offset; + int gs_out_ring_offset; + int gs_next_vertex; + struct r600_shader *gs_for_vs; + int gs_export_gpr_treg; }; struct r600_shader_tgsi_instruction { @@ -254,6 +302,7 @@ struct r600_shader_tgsi_instruction { int (*process)(struct r600_shader_ctx *ctx); }; +static int emit_gs_ring_writes(struct r600_shader_ctx *ctx, bool ind); static struct r600_shader_tgsi_instruction r600_shader_tgsi_instruction[], eg_shader_tgsi_instruction[], cm_shader_tgsi_instruction[]; static int tgsi_helper_tempx_replicate(struct r600_shader_ctx *ctx); static inline void callstack_push(struct r600_shader_ctx *ctx, unsigned reason); @@ -285,7 +334,13 @@ static int tgsi_is_supported(struct r600_shader_ctx *ctx) #endif for (j = 0; j < i->Instruction.NumSrcRegs; j++) { if (i->Src[j].Register.Dimension) { - if (i->Src[j].Register.File != TGSI_FILE_CONSTANT) { + switch (i->Src[j].Register.File) { + case TGSI_FILE_CONSTANT: + break; + case TGSI_FILE_INPUT: + if (ctx->type == TGSI_PROCESSOR_GEOMETRY) + break; + default: R600_ERR("unsupported src %d (dimension %d)\n", j, i->Src[j].Register.Dimension); return -EINVAL; @@ -436,6 +491,7 @@ static int r600_spi_sid(struct r600_shader_io * io) */ if (name == TGSI_SEMANTIC_POSITION || name == TGSI_SEMANTIC_PSIZE || + name == TGSI_SEMANTIC_LAYER || name == TGSI_SEMANTIC_FACE) index = 0; else { @@ -536,6 +592,12 @@ static int tgsi_declaration(struct r600_shader_ctx *ctx) if ((r = evergreen_interp_input(ctx, i))) return r; } + } else if (ctx->type == TGSI_PROCESSOR_GEOMETRY) { + /* FIXME probably skip inputs if they aren't passed in the ring */ + ctx->shader->input[i].ring_offset = ctx->next_ring_offset; + ctx->next_ring_offset += 16; + if (ctx->shader->input[i].name == TGSI_SEMANTIC_PRIMID) + ctx->shader->gs_prim_id_input = true; } for (j = 1; j < count; ++j) { ctx->shader->input[i + j] = ctx->shader->input[i]; @@ -550,7 +612,8 @@ static int tgsi_declaration(struct r600_shader_ctx *ctx) ctx->shader->output[i].gpr = ctx->file_offset[TGSI_FILE_OUTPUT] + d->Range.First; ctx->shader->output[i].interpolate = d->Interp.Interpolate; ctx->shader->output[i].write_mask = d->Declaration.UsageMask; - if (ctx->type == TGSI_PROCESSOR_VERTEX) { + if (ctx->type == TGSI_PROCESSOR_VERTEX || + ctx->type == TGSI_PROCESSOR_GEOMETRY) { ctx->shader->output[i].spi_sid = r600_spi_sid(&ctx->shader->output[i]); switch (d->Semantic.Name) { case TGSI_SEMANTIC_CLIPDIST: @@ -560,11 +623,18 @@ static int tgsi_declaration(struct r600_shader_ctx *ctx) ctx->shader->vs_out_misc_write = 1; ctx->shader->vs_out_point_size = 1; break; + case TGSI_SEMANTIC_LAYER: + ctx->shader->vs_out_misc_write = 1; + ctx->shader->vs_out_layer = 1; + break; case TGSI_SEMANTIC_CLIPVERTEX: ctx->clip_vertex_write = TRUE; ctx->cv_output = i; break; } + if (ctx->type == TGSI_PROCESSOR_GEOMETRY) { + ctx->gs_out_ring_offset += 16; + } } else if (ctx->type == TGSI_PROCESSOR_FRAGMENT) { switch (d->Semantic.Name) { case TGSI_SEMANTIC_COLOR: @@ -773,6 +843,110 @@ static int tgsi_fetch_rel_const(struct r600_shader_ctx *ctx, unsigned int cb_idx return 0; } +static int fetch_gs_input(struct r600_shader_ctx *ctx, struct tgsi_full_src_register *src, unsigned int dst_reg) +{ + struct r600_bytecode_vtx vtx; + int r; + unsigned index = src->Register.Index; + unsigned vtx_id = src->Dimension.Index; + int offset_reg = vtx_id / 3; + int offset_chan = vtx_id % 3; + + /* offsets of per-vertex data in ESGS ring are passed to GS in R0.x, R0.y, + * R0.w, R1.x, R1.y, R1.z (it seems R0.z is used for PrimitiveID) */ + + if (offset_reg == 0 && offset_chan == 2) + offset_chan = 3; + + if (src->Dimension.Indirect) { + int treg[3]; + int t2; + struct r600_bytecode_alu alu; + int r, i; + + /* you have got to be shitting me - + we have to put the R0.x/y/w into Rt.x Rt+1.x Rt+2.x then index reg from Rt. + at least this is what fglrx seems to do. */ + for (i = 0; i < 3; i++) { + treg[i] = r600_get_temp(ctx); + } + t2 = r600_get_temp(ctx); + for (i = 0; i < 3; i++) { + memset(&alu, 0, sizeof(struct r600_bytecode_alu)); + alu.op = ALU_OP1_MOV; + alu.src[0].sel = 0; + alu.src[0].chan = i == 2 ? 3 : i; + alu.dst.sel = treg[i]; + alu.dst.chan = 0; + alu.dst.write = 1; + alu.last = 1; + r = r600_bytecode_add_alu(ctx->bc, &alu); + if (r) + return r; + } + memset(&alu, 0, sizeof(struct r600_bytecode_alu)); + alu.op = ALU_OP1_MOV; + alu.src[0].sel = treg[0]; + alu.src[0].rel = 1; + alu.dst.sel = t2; + alu.dst.write = 1; + alu.last = 1; + r = r600_bytecode_add_alu(ctx->bc, &alu); + if (r) + return r; + offset_reg = t2; + } + + + memset(&vtx, 0, sizeof(vtx)); + vtx.buffer_id = R600_GS_RING_CONST_BUFFER; + vtx.fetch_type = 2; /* VTX_FETCH_NO_INDEX_OFFSET */ + vtx.src_gpr = offset_reg; + vtx.src_sel_x = offset_chan; + vtx.offset = index * 16; /*bytes*/ + vtx.mega_fetch_count = 16; + vtx.dst_gpr = dst_reg; + vtx.dst_sel_x = 0; /* SEL_X */ + vtx.dst_sel_y = 1; /* SEL_Y */ + vtx.dst_sel_z = 2; /* SEL_Z */ + vtx.dst_sel_w = 3; /* SEL_W */ + if (ctx->bc->chip_class >= EVERGREEN) { + vtx.use_const_fields = 1; + } else { + vtx.data_format = FMT_32_32_32_32_FLOAT; + } + + if ((r = r600_bytecode_add_vtx(ctx->bc, &vtx))) + return r; + + return 0; +} + +static int tgsi_split_gs_inputs(struct r600_shader_ctx *ctx) +{ + struct tgsi_full_instruction *inst = &ctx->parse.FullToken.FullInstruction; + int i; + + for (i = 0; i < inst->Instruction.NumSrcRegs; i++) { + struct tgsi_full_src_register *src = &inst->Src[i]; + + if (src->Register.File == TGSI_FILE_INPUT) { + if (ctx->shader->input[src->Register.Index].name == TGSI_SEMANTIC_PRIMID) { + /* primitive id is in R0.z */ + ctx->src[i].sel = 0; + ctx->src[i].swizzle[0] = 2; + } + } + if (src->Register.File == TGSI_FILE_INPUT && src->Register.Dimension) { + int treg = r600_get_temp(ctx); + + fetch_gs_input(ctx, src, treg); + ctx->src[i].sel = treg; + } + } + return 0; +} + static int tgsi_split_constant(struct r600_shader_ctx *ctx) { struct tgsi_full_instruction *inst = &ctx->parse.FullToken.FullInstruction; @@ -875,11 +1049,418 @@ static int process_twoside_color_inputs(struct r600_shader_ctx *ctx) return 0; } +static int emit_streamout(struct r600_shader_ctx *ctx, struct pipe_stream_output_info *so) +{ + unsigned so_gpr[PIPE_MAX_SHADER_OUTPUTS]; + int i, j, r; + + /* Sanity checking. */ + if (so->num_outputs > PIPE_MAX_SHADER_OUTPUTS) { + R600_ERR("Too many stream outputs: %d\n", so->num_outputs); + r = -EINVAL; + goto out_err; + } + for (i = 0; i < so->num_outputs; i++) { + if (so->output[i].output_buffer >= 4) { + R600_ERR("Exceeded the max number of stream output buffers, got: %d\n", + so->output[i].output_buffer); + r = -EINVAL; + goto out_err; + } + } -static int r600_shader_from_tgsi(struct r600_screen *rscreen, + /* Initialize locations where the outputs are stored. */ + for (i = 0; i < so->num_outputs; i++) { + so_gpr[i] = ctx->shader->output[so->output[i].register_index].gpr; + + /* Lower outputs with dst_offset < start_component. + * + * We can only output 4D vectors with a write mask, e.g. we can + * only output the W component at offset 3, etc. If we want + * to store Y, Z, or W at buffer offset 0, we need to use MOV + * to move it to X and output X. */ + if (so->output[i].dst_offset < so->output[i].start_component) { + unsigned tmp = r600_get_temp(ctx); + + for (j = 0; j < so->output[i].num_components; j++) { + struct r600_bytecode_alu alu; + memset(&alu, 0, sizeof(struct r600_bytecode_alu)); + alu.op = ALU_OP1_MOV; + alu.src[0].sel = so_gpr[i]; + alu.src[0].chan = so->output[i].start_component + j; + + alu.dst.sel = tmp; + alu.dst.chan = j; + alu.dst.write = 1; + if (j == so->output[i].num_components - 1) + alu.last = 1; + r = r600_bytecode_add_alu(ctx->bc, &alu); + if (r) + return r; + } + so->output[i].start_component = 0; + so_gpr[i] = tmp; + } + } + + /* Write outputs to buffers. */ + for (i = 0; i < so->num_outputs; i++) { + struct r600_bytecode_output output; + + memset(&output, 0, sizeof(struct r600_bytecode_output)); + output.gpr = so_gpr[i]; + output.elem_size = so->output[i].num_components; + output.array_base = so->output[i].dst_offset - so->output[i].start_component; + output.type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_WRITE; + output.burst_count = 1; + /* array_size is an upper limit for the burst_count + * with MEM_STREAM instructions */ + output.array_size = 0xFFF; + output.comp_mask = ((1 << so->output[i].num_components) - 1) << so->output[i].start_component; + if (ctx->bc->chip_class >= EVERGREEN) { + switch (so->output[i].output_buffer) { + case 0: + output.op = CF_OP_MEM_STREAM0_BUF0; + break; + case 1: + output.op = CF_OP_MEM_STREAM0_BUF1; + break; + case 2: + output.op = CF_OP_MEM_STREAM0_BUF2; + break; + case 3: + output.op = CF_OP_MEM_STREAM0_BUF3; + break; + } + } else { + switch (so->output[i].output_buffer) { + case 0: + output.op = CF_OP_MEM_STREAM0; + break; + case 1: + output.op = CF_OP_MEM_STREAM1; + break; + case 2: + output.op = CF_OP_MEM_STREAM2; + break; + case 3: + output.op = CF_OP_MEM_STREAM3; + break; + } + } + r = r600_bytecode_add_output(ctx->bc, &output); + if (r) + goto out_err; + } + return 0; +out_err: + return r; +} + +static int generate_gs_copy_shader(struct r600_context *rctx, + struct r600_pipe_shader *gs, + struct pipe_stream_output_info *so) +{ + struct r600_shader_ctx ctx = {}; + struct r600_shader *gs_shader = &gs->shader; + struct r600_pipe_shader *cshader; + int ocnt = gs_shader->noutput; + struct r600_bytecode_alu alu; + struct r600_bytecode_vtx vtx; + struct r600_bytecode_output output; + struct r600_bytecode_cf *cf_jump, *cf_pop, + *last_exp_pos = NULL, *last_exp_param = NULL; + int i, next_clip_pos = 61, next_param = 0; + + cshader = calloc(1, sizeof(struct r600_pipe_shader)); + if (!cshader) + return 0; + + memcpy(cshader->shader.output, gs_shader->output, ocnt * + sizeof(struct r600_shader_io)); + + cshader->shader.noutput = ocnt; + + ctx.shader = &cshader->shader; + ctx.bc = &ctx.shader->bc; + ctx.type = ctx.bc->type = TGSI_PROCESSOR_VERTEX; + + r600_bytecode_init(ctx.bc, rctx->b.chip_class, rctx->b.family, + rctx->screen->has_compressed_msaa_texturing); + + ctx.bc->isa = rctx->isa; + + /* R0.x = R0.x & 0x3fffffff */ + memset(&alu, 0, sizeof(alu)); + alu.op = ALU_OP2_AND_INT; + alu.src[1].sel = V_SQ_ALU_SRC_LITERAL; + alu.src[1].value = 0x3fffffff; + alu.dst.write = 1; + r600_bytecode_add_alu(ctx.bc, &alu); + + /* R0.y = R0.x >> 30 */ + memset(&alu, 0, sizeof(alu)); + alu.op = ALU_OP2_LSHR_INT; + alu.src[1].sel = V_SQ_ALU_SRC_LITERAL; + alu.src[1].value = 0x1e; + alu.dst.chan = 1; + alu.dst.write = 1; + alu.last = 1; + r600_bytecode_add_alu(ctx.bc, &alu); + + /* PRED_SETE_INT __, R0.y, 0 */ + memset(&alu, 0, sizeof(alu)); + alu.op = ALU_OP2_PRED_SETE_INT; + alu.src[0].chan = 1; + alu.src[1].sel = V_SQ_ALU_SRC_0; + alu.execute_mask = 1; + alu.update_pred = 1; + alu.last = 1; + r600_bytecode_add_alu_type(ctx.bc, &alu, CF_OP_ALU_PUSH_BEFORE); + + r600_bytecode_add_cfinst(ctx.bc, CF_OP_JUMP); + cf_jump = ctx.bc->cf_last; + + /* fetch vertex data from GSVS ring */ + for (i = 0; i < ocnt; ++i) { + struct r600_shader_io *out = &ctx.shader->output[i]; + out->gpr = i + 1; + out->ring_offset = i * 16; + + memset(&vtx, 0, sizeof(vtx)); + vtx.op = FETCH_OP_VFETCH; + vtx.buffer_id = R600_GS_RING_CONST_BUFFER; + vtx.fetch_type = 2; + vtx.offset = out->ring_offset; + vtx.dst_gpr = out->gpr; + vtx.dst_sel_x = 0; + vtx.dst_sel_y = 1; + vtx.dst_sel_z = 2; + vtx.dst_sel_w = 3; + if (rctx->b.chip_class >= EVERGREEN) { + vtx.use_const_fields = 1; + } else { + vtx.data_format = FMT_32_32_32_32_FLOAT; + } + + r600_bytecode_add_vtx(ctx.bc, &vtx); + } + + /* XXX handle clipvertex, streamout? */ + emit_streamout(&ctx, so); + + /* export vertex data */ + /* XXX factor out common code with r600_shader_from_tgsi ? */ + for (i = 0; i < ocnt; ++i) { + struct r600_shader_io *out = &ctx.shader->output[i]; + + if (out->name == TGSI_SEMANTIC_CLIPVERTEX) + continue; + + memset(&output, 0, sizeof(output)); + output.gpr = out->gpr; + output.elem_size = 3; + output.swizzle_x = 0; + output.swizzle_y = 1; + output.swizzle_z = 2; + output.swizzle_w = 3; + output.burst_count = 1; + output.type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_PARAM; + output.op = CF_OP_EXPORT; + switch (out->name) { + case TGSI_SEMANTIC_POSITION: + output.array_base = 60; + output.type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_POS; + break; + + case TGSI_SEMANTIC_PSIZE: + output.array_base = 61; + if (next_clip_pos == 61) + next_clip_pos = 62; + output.type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_POS; + output.swizzle_y = 7; + output.swizzle_z = 7; + output.swizzle_w = 7; + ctx.shader->vs_out_misc_write = 1; + ctx.shader->vs_out_point_size = 1; + break; + case TGSI_SEMANTIC_LAYER: + output.array_base = 61; + if (next_clip_pos == 61) + next_clip_pos = 62; + output.type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_POS; + output.swizzle_x = 7; + output.swizzle_y = 7; + output.swizzle_z = 0; + output.swizzle_w = 7; + ctx.shader->vs_out_misc_write = 1; + ctx.shader->vs_out_layer = 1; + break; + case TGSI_SEMANTIC_CLIPDIST: + /* spi_sid is 0 for clipdistance outputs that were generated + * for clipvertex - we don't need to pass them to PS */ + ctx.shader->clip_dist_write = gs->shader.clip_dist_write; + if (out->spi_sid) { + /* duplicate it as PARAM to pass to the pixel shader */ + output.array_base = next_param++; + r600_bytecode_add_output(ctx.bc, &output); + last_exp_param = ctx.bc->cf_last; + } + output.array_base = next_clip_pos++; + output.type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_POS; + break; + case TGSI_SEMANTIC_FOG: + output.swizzle_y = 4; /* 0 */ + output.swizzle_z = 4; /* 0 */ + output.swizzle_w = 5; /* 1 */ + break; + default: + output.array_base = next_param++; + break; + } + r600_bytecode_add_output(ctx.bc, &output); + if (output.type == V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_PARAM) + last_exp_param = ctx.bc->cf_last; + else + last_exp_pos = ctx.bc->cf_last; + } + + if (!last_exp_pos) { + memset(&output, 0, sizeof(output)); + output.gpr = 0; + output.elem_size = 3; + output.swizzle_x = 7; + output.swizzle_y = 7; + output.swizzle_z = 7; + output.swizzle_w = 7; + output.burst_count = 1; + output.type = 2; + output.op = CF_OP_EXPORT; + output.array_base = 60; + output.type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_POS; + r600_bytecode_add_output(ctx.bc, &output); + last_exp_pos = ctx.bc->cf_last; + } + + if (!last_exp_param) { + memset(&output, 0, sizeof(output)); + output.gpr = 0; + output.elem_size = 3; + output.swizzle_x = 7; + output.swizzle_y = 7; + output.swizzle_z = 7; + output.swizzle_w = 7; + output.burst_count = 1; + output.type = 2; + output.op = CF_OP_EXPORT; + output.array_base = next_param++; + output.type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_PARAM; + r600_bytecode_add_output(ctx.bc, &output); + last_exp_param = ctx.bc->cf_last; + } + + last_exp_pos->op = CF_OP_EXPORT_DONE; + last_exp_param->op = CF_OP_EXPORT_DONE; + + r600_bytecode_add_cfinst(ctx.bc, CF_OP_POP); + cf_pop = ctx.bc->cf_last; + + cf_jump->cf_addr = cf_pop->id + 2; + cf_jump->pop_count = 1; + cf_pop->cf_addr = cf_pop->id + 2; + cf_pop->pop_count = 1; + + if (ctx.bc->chip_class == CAYMAN) + cm_bytecode_add_cf_end(ctx.bc); + else { + r600_bytecode_add_cfinst(ctx.bc, CF_OP_NOP); + ctx.bc->cf_last->end_of_program = 1; + } + + gs->gs_copy_shader = cshader; + + ctx.bc->nstack = 1; + cshader->shader.ring_item_size = ocnt * 16; + + return r600_bytecode_build(ctx.bc); +} + +static int emit_gs_ring_writes(struct r600_shader_ctx *ctx, bool ind) +{ + struct r600_bytecode_output output; + int i, k, ring_offset; + + for (i = 0; i < ctx->shader->noutput; i++) { + if (ctx->gs_for_vs) { + /* for ES we need to lookup corresponding ring offset expected by GS + * (map this output to GS input by name and sid) */ + /* FIXME precompute offsets */ + ring_offset = -1; + for(k = 0; k < ctx->gs_for_vs->ninput; ++k) { + struct r600_shader_io *in = &ctx->gs_for_vs->input[k]; + struct r600_shader_io *out = &ctx->shader->output[i]; + if (in->name == out->name && in->sid == out->sid) + ring_offset = in->ring_offset; + } + + if (ring_offset == -1) + continue; + } else + ring_offset = i * 16; + + /* next_ring_offset after parsing input decls contains total size of + * single vertex data, gs_next_vertex - current vertex index */ + if (!ind) + ring_offset += ctx->gs_out_ring_offset * ctx->gs_next_vertex; + + /* get a temp and add the ring offset to the next vertex base in the shader */ + memset(&output, 0, sizeof(struct r600_bytecode_output)); + output.gpr = ctx->shader->output[i].gpr; + output.elem_size = 3; + output.comp_mask = 0xF; + output.burst_count = 1; + + if (ind) + output.type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_WRITE_IND; + else + output.type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_WRITE; + output.op = CF_OP_MEM_RING; + + + if (ind) { + output.array_base = ring_offset >> 2; /* in dwords */ + output.array_size = 0xfff; + output.index_gpr = ctx->gs_export_gpr_treg; + } else + output.array_base = ring_offset >> 2; /* in dwords */ + r600_bytecode_add_output(ctx->bc, &output); + } + + if (ind) { + struct r600_bytecode_alu alu; + int r; + + memset(&alu, 0, sizeof(struct r600_bytecode_alu)); + alu.op = ALU_OP2_ADD_INT; + alu.src[0].sel = ctx->gs_export_gpr_treg; + alu.src[1].sel = V_SQ_ALU_SRC_LITERAL; + alu.src[1].value = ctx->gs_out_ring_offset >> 4; + alu.dst.sel = ctx->gs_export_gpr_treg; + alu.dst.write = 1; + alu.last = 1; + r = r600_bytecode_add_alu(ctx->bc, &alu); + if (r) + return r; + } + ++ctx->gs_next_vertex; + return 0; +} + +static int r600_shader_from_tgsi(struct r600_context *rctx, struct r600_pipe_shader *pipeshader, struct r600_shader_key key) { + struct r600_screen *rscreen = rctx->screen; struct r600_shader *shader = &pipeshader->shader; struct tgsi_token *tokens = pipeshader->selector->tokens; struct pipe_stream_output_info so = pipeshader->selector->so; @@ -890,11 +1471,13 @@ static int r600_shader_from_tgsi(struct r600_screen *rscreen, unsigned output_done, noutput; unsigned opcode; int i, j, k, r = 0; - int next_pos_base = 60, next_param_base = 0; + int next_param_base = 0, next_clip_base; int max_color_exports = MAX2(key.nr_cbufs, 1); /* Declarations used by llvm code */ bool use_llvm = false; bool indirect_gprs; + bool ring_outputs = false; + bool pos_emitted = false; #ifdef R600_USE_LLVM use_llvm = !(rscreen->b.debug_flags & DBG_NO_LLVM); @@ -903,6 +1486,8 @@ static int r600_shader_from_tgsi(struct r600_screen *rscreen, ctx.shader = shader; ctx.native_integers = true; + shader->vs_as_es = key.vs_as_es; + r600_bytecode_init(ctx.bc, rscreen->b.chip_class, rscreen->b.family, rscreen->has_compressed_msaa_texturing); ctx.tokens = tokens; @@ -914,6 +1499,18 @@ static int r600_shader_from_tgsi(struct r600_screen *rscreen, shader->processor_type = ctx.type; ctx.bc->type = shader->processor_type; + ring_outputs = key.vs_as_es || (ctx.type == TGSI_PROCESSOR_GEOMETRY); + + if (key.vs_as_es) { + ctx.gs_for_vs = &rctx->gs_shader->current->shader; + } else { + ctx.gs_for_vs = NULL; + } + + ctx.next_ring_offset = 0; + ctx.gs_out_ring_offset = 0; + ctx.gs_next_vertex = 0; + ctx.face_gpr = -1; ctx.fragcoord_input = -1; ctx.colors_used = 0; @@ -966,6 +1563,10 @@ static int r600_shader_from_tgsi(struct r600_screen *rscreen, if (ctx.type == TGSI_PROCESSOR_FRAGMENT && ctx.bc->chip_class >= EVERGREEN) { ctx.file_offset[TGSI_FILE_INPUT] = evergreen_gpr_count(&ctx); } + if (ctx.type == TGSI_PROCESSOR_GEOMETRY) { + /* FIXME 1 would be enough in some cases (3 or less input vertices) */ + ctx.file_offset[TGSI_FILE_INPUT] = 2; + } ctx.use_llvm = use_llvm; if (use_llvm) { @@ -986,7 +1587,11 @@ static int r600_shader_from_tgsi(struct r600_screen *rscreen, ctx.file_offset[TGSI_FILE_IMMEDIATE] = V_SQ_ALU_SRC_LITERAL; ctx.bc->ar_reg = ctx.file_offset[TGSI_FILE_TEMPORARY] + ctx.info.file_max[TGSI_FILE_TEMPORARY] + 1; - ctx.temp_reg = ctx.bc->ar_reg + 1; + if (ctx.type == TGSI_PROCESSOR_GEOMETRY) { + ctx.gs_export_gpr_treg = ctx.bc->ar_reg + 1; + ctx.temp_reg = ctx.bc->ar_reg + 2; + } else + ctx.temp_reg = ctx.bc->ar_reg + 1; if (indirect_gprs) { shader->max_arrays = 0; @@ -1042,6 +1647,15 @@ static int r600_shader_from_tgsi(struct r600_screen *rscreen, case TGSI_PROPERTY_VS_PROHIBIT_UCPS: /* we don't need this one */ break; + case TGSI_PROPERTY_GS_INPUT_PRIM: + shader->gs_input_prim = property->u[0].Data; + break; + case TGSI_PROPERTY_GS_OUTPUT_PRIM: + shader->gs_output_prim = property->u[0].Data; + break; + case TGSI_PROPERTY_GS_MAX_OUTPUT_VERTICES: + shader->gs_max_out_vertices = property->u[0].Data; + break; } break; default: @@ -1051,6 +1665,8 @@ static int r600_shader_from_tgsi(struct r600_screen *rscreen, } } + shader->ring_item_size = ctx.next_ring_offset; + /* Process two side if needed */ if (shader->two_side && ctx.colors_used) { int i, count = ctx.shader->ninput; @@ -1169,6 +1785,21 @@ static int r600_shader_from_tgsi(struct r600_screen *rscreen, } } + if (ctx.type == TGSI_PROCESSOR_GEOMETRY) { + struct r600_bytecode_alu alu; + int r; + + memset(&alu, 0, sizeof(struct r600_bytecode_alu)); + alu.op = ALU_OP1_MOV; + alu.src[0].sel = V_SQ_ALU_SRC_LITERAL; + alu.src[0].value = 0; + alu.dst.sel = ctx.gs_export_gpr_treg; + alu.dst.write = 1; + alu.last = 1; + r = r600_bytecode_add_alu(ctx.bc, &alu); + if (r) + return r; + } if (shader->two_side && ctx.colors_used) { if ((r = process_twoside_color_inputs(&ctx))) return r; @@ -1191,6 +1822,9 @@ static int r600_shader_from_tgsi(struct r600_screen *rscreen, goto out_err; if ((r = tgsi_split_literal_constant(&ctx))) goto out_err; + if (ctx.type == TGSI_PROCESSOR_GEOMETRY) + if ((r = tgsi_split_gs_inputs(&ctx))) + goto out_err; if (ctx.bc->chip_class == CAYMAN) ctx.inst_info = &cm_shader_tgsi_instruction[opcode]; else if (ctx.bc->chip_class >= EVERGREEN) @@ -1212,7 +1846,7 @@ static int r600_shader_from_tgsi(struct r600_screen *rscreen, noutput = shader->noutput; - if (ctx.clip_vertex_write) { + if (!ring_outputs && ctx.clip_vertex_write) { unsigned clipdist_temp[2]; clipdist_temp[0] = r600_get_temp(&ctx); @@ -1263,220 +1897,139 @@ static int r600_shader_from_tgsi(struct r600_screen *rscreen, } /* Add stream outputs. */ - if (ctx.type == TGSI_PROCESSOR_VERTEX && so.num_outputs && !use_llvm) { - unsigned so_gpr[PIPE_MAX_SHADER_OUTPUTS]; - - /* Sanity checking. */ - if (so.num_outputs > PIPE_MAX_SHADER_OUTPUTS) { - R600_ERR("Too many stream outputs: %d\n", so.num_outputs); - r = -EINVAL; - goto out_err; - } - for (i = 0; i < so.num_outputs; i++) { - if (so.output[i].output_buffer >= 4) { - R600_ERR("Exceeded the max number of stream output buffers, got: %d\n", - so.output[i].output_buffer); - r = -EINVAL; - goto out_err; - } - } - - /* Initialize locations where the outputs are stored. */ - for (i = 0; i < so.num_outputs; i++) { - so_gpr[i] = shader->output[so.output[i].register_index].gpr; - - /* Lower outputs with dst_offset < start_component. - * - * We can only output 4D vectors with a write mask, e.g. we can - * only output the W component at offset 3, etc. If we want - * to store Y, Z, or W at buffer offset 0, we need to use MOV - * to move it to X and output X. */ - if (so.output[i].dst_offset < so.output[i].start_component) { - unsigned tmp = r600_get_temp(&ctx); - - for (j = 0; j < so.output[i].num_components; j++) { - struct r600_bytecode_alu alu; - memset(&alu, 0, sizeof(struct r600_bytecode_alu)); - alu.op = ALU_OP1_MOV; - alu.src[0].sel = so_gpr[i]; - alu.src[0].chan = so.output[i].start_component + j; - - alu.dst.sel = tmp; - alu.dst.chan = j; - alu.dst.write = 1; - if (j == so.output[i].num_components - 1) - alu.last = 1; - r = r600_bytecode_add_alu(ctx.bc, &alu); - if (r) - return r; - } - so.output[i].start_component = 0; - so_gpr[i] = tmp; - } - } + if (!ring_outputs && ctx.type == TGSI_PROCESSOR_VERTEX && + so.num_outputs && !use_llvm) + emit_streamout(&ctx, &so); - /* Write outputs to buffers. */ - for (i = 0; i < so.num_outputs; i++) { - struct r600_bytecode_output output; + if (ring_outputs) { + if (key.vs_as_es) + emit_gs_ring_writes(&ctx, FALSE); + } else { + /* Export output */ + next_clip_base = shader->vs_out_misc_write ? 62 : 61; - memset(&output, 0, sizeof(struct r600_bytecode_output)); - output.gpr = so_gpr[i]; - output.elem_size = so.output[i].num_components; - output.array_base = so.output[i].dst_offset - so.output[i].start_component; - output.type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_WRITE; - output.burst_count = 1; - output.barrier = 1; - /* array_size is an upper limit for the burst_count - * with MEM_STREAM instructions */ - output.array_size = 0xFFF; - output.comp_mask = ((1 << so.output[i].num_components) - 1) << so.output[i].start_component; - if (ctx.bc->chip_class >= EVERGREEN) { - switch (so.output[i].output_buffer) { - case 0: - output.op = CF_OP_MEM_STREAM0_BUF0; - break; - case 1: - output.op = CF_OP_MEM_STREAM0_BUF1; - break; - case 2: - output.op = CF_OP_MEM_STREAM0_BUF2; + for (i = 0, j = 0; i < noutput; i++, j++) { + memset(&output[j], 0, sizeof(struct r600_bytecode_output)); + output[j].gpr = shader->output[i].gpr; + output[j].elem_size = 3; + output[j].swizzle_x = 0; + output[j].swizzle_y = 1; + output[j].swizzle_z = 2; + output[j].swizzle_w = 3; + output[j].burst_count = 1; + output[j].type = -1; + output[j].op = CF_OP_EXPORT; + switch (ctx.type) { + case TGSI_PROCESSOR_VERTEX: + switch (shader->output[i].name) { + case TGSI_SEMANTIC_POSITION: + output[j].array_base = 60; + output[j].type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_POS; + pos_emitted = true; break; - case 3: - output.op = CF_OP_MEM_STREAM0_BUF3; + + case TGSI_SEMANTIC_PSIZE: + output[j].array_base = 61; + output[j].swizzle_y = 7; + output[j].swizzle_z = 7; + output[j].swizzle_w = 7; + output[j].type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_POS; + pos_emitted = true; break; - } - } else { - switch (so.output[i].output_buffer) { - case 0: - output.op = CF_OP_MEM_STREAM0; + case TGSI_SEMANTIC_LAYER: + output[j].array_base = 61; + output[j].swizzle_x = 7; + output[j].swizzle_y = 7; + output[j].swizzle_z = 0; + output[j].swizzle_w = 7; + output[j].type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_POS; + pos_emitted = true; break; - case 1: - output.op = CF_OP_MEM_STREAM1; + case TGSI_SEMANTIC_CLIPVERTEX: + j--; break; - case 2: - output.op = CF_OP_MEM_STREAM2; + case TGSI_SEMANTIC_CLIPDIST: + output[j].array_base = next_clip_base++; + output[j].type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_POS; + pos_emitted = true; + /* spi_sid is 0 for clipdistance outputs that were generated + * for clipvertex - we don't need to pass them to PS */ + if (shader->output[i].spi_sid) { + j++; + /* duplicate it as PARAM to pass to the pixel shader */ + memcpy(&output[j], &output[j-1], sizeof(struct r600_bytecode_output)); + output[j].array_base = next_param_base++; + output[j].type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_PARAM; + } break; - case 3: - output.op = CF_OP_MEM_STREAM3; + case TGSI_SEMANTIC_FOG: + output[j].swizzle_y = 4; /* 0 */ + output[j].swizzle_z = 4; /* 0 */ + output[j].swizzle_w = 5; /* 1 */ break; } - } - r = r600_bytecode_add_output(ctx.bc, &output); - if (r) - goto out_err; - } - } - - /* export output */ - for (i = 0, j = 0; i < noutput; i++, j++) { - memset(&output[j], 0, sizeof(struct r600_bytecode_output)); - output[j].gpr = shader->output[i].gpr; - output[j].elem_size = 3; - output[j].swizzle_x = 0; - output[j].swizzle_y = 1; - output[j].swizzle_z = 2; - output[j].swizzle_w = 3; - output[j].burst_count = 1; - output[j].barrier = 1; - output[j].type = -1; - output[j].op = CF_OP_EXPORT; - switch (ctx.type) { - case TGSI_PROCESSOR_VERTEX: - switch (shader->output[i].name) { - case TGSI_SEMANTIC_POSITION: - output[j].array_base = next_pos_base++; - output[j].type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_POS; - break; - - case TGSI_SEMANTIC_PSIZE: - output[j].array_base = next_pos_base++; - output[j].type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_POS; - break; - case TGSI_SEMANTIC_CLIPVERTEX: - j--; - break; - case TGSI_SEMANTIC_CLIPDIST: - output[j].array_base = next_pos_base++; - output[j].type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_POS; - /* spi_sid is 0 for clipdistance outputs that were generated - * for clipvertex - we don't need to pass them to PS */ - if (shader->output[i].spi_sid) { - j++; - /* duplicate it as PARAM to pass to the pixel shader */ - memcpy(&output[j], &output[j-1], sizeof(struct r600_bytecode_output)); - output[j].array_base = next_param_base++; - output[j].type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_PARAM; - } - break; - case TGSI_SEMANTIC_FOG: - output[j].swizzle_y = 4; /* 0 */ - output[j].swizzle_z = 4; /* 0 */ - output[j].swizzle_w = 5; /* 1 */ break; - } - break; - case TGSI_PROCESSOR_FRAGMENT: - if (shader->output[i].name == TGSI_SEMANTIC_COLOR) { - /* never export more colors than the number of CBs */ - if (shader->output[i].sid >= max_color_exports) { - /* skip export */ - j--; - continue; - } - output[j].swizzle_w = key.alpha_to_one ? 5 : 3; - output[j].array_base = shader->output[i].sid; - output[j].type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_PIXEL; - shader->nr_ps_color_exports++; - if (shader->fs_write_all && (rscreen->b.chip_class >= EVERGREEN)) { - for (k = 1; k < max_color_exports; k++) { - j++; - memset(&output[j], 0, sizeof(struct r600_bytecode_output)); - output[j].gpr = shader->output[i].gpr; - output[j].elem_size = 3; - output[j].swizzle_x = 0; - output[j].swizzle_y = 1; - output[j].swizzle_z = 2; - output[j].swizzle_w = key.alpha_to_one ? 5 : 3; - output[j].burst_count = 1; - output[j].barrier = 1; - output[j].array_base = k; - output[j].op = CF_OP_EXPORT; - output[j].type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_PIXEL; - shader->nr_ps_color_exports++; + case TGSI_PROCESSOR_FRAGMENT: + if (shader->output[i].name == TGSI_SEMANTIC_COLOR) { + /* never export more colors than the number of CBs */ + if (shader->output[i].sid >= max_color_exports) { + /* skip export */ + j--; + continue; + } + output[j].swizzle_w = key.alpha_to_one ? 5 : 3; + output[j].array_base = shader->output[i].sid; + output[j].type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_PIXEL; + shader->nr_ps_color_exports++; + if (shader->fs_write_all && (rscreen->b.chip_class >= EVERGREEN)) { + for (k = 1; k < max_color_exports; k++) { + j++; + memset(&output[j], 0, sizeof(struct r600_bytecode_output)); + output[j].gpr = shader->output[i].gpr; + output[j].elem_size = 3; + output[j].swizzle_x = 0; + output[j].swizzle_y = 1; + output[j].swizzle_z = 2; + output[j].swizzle_w = key.alpha_to_one ? 5 : 3; + output[j].burst_count = 1; + output[j].array_base = k; + output[j].op = CF_OP_EXPORT; + output[j].type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_PIXEL; + shader->nr_ps_color_exports++; + } } + } else if (shader->output[i].name == TGSI_SEMANTIC_POSITION) { + output[j].array_base = 61; + output[j].swizzle_x = 2; + output[j].swizzle_y = 7; + output[j].swizzle_z = output[j].swizzle_w = 7; + output[j].type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_PIXEL; + } else if (shader->output[i].name == TGSI_SEMANTIC_STENCIL) { + output[j].array_base = 61; + output[j].swizzle_x = 7; + output[j].swizzle_y = 1; + output[j].swizzle_z = output[j].swizzle_w = 7; + output[j].type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_PIXEL; + } else { + R600_ERR("unsupported fragment output name %d\n", shader->output[i].name); + r = -EINVAL; + goto out_err; } - } else if (shader->output[i].name == TGSI_SEMANTIC_POSITION) { - output[j].array_base = 61; - output[j].swizzle_x = 2; - output[j].swizzle_y = 7; - output[j].swizzle_z = output[j].swizzle_w = 7; - output[j].type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_PIXEL; - } else if (shader->output[i].name == TGSI_SEMANTIC_STENCIL) { - output[j].array_base = 61; - output[j].swizzle_x = 7; - output[j].swizzle_y = 1; - output[j].swizzle_z = output[j].swizzle_w = 7; - output[j].type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_PIXEL; - } else { - R600_ERR("unsupported fragment output name %d\n", shader->output[i].name); + break; + default: + R600_ERR("unsupported processor type %d\n", ctx.type); r = -EINVAL; goto out_err; } - break; - default: - R600_ERR("unsupported processor type %d\n", ctx.type); - r = -EINVAL; - goto out_err; - } - if (output[j].type==-1) { - output[j].type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_PARAM; - output[j].array_base = next_param_base++; + if (output[j].type==-1) { + output[j].type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_PARAM; + output[j].array_base = next_param_base++; + } } - } - /* add fake position export */ - if (ctx.type == TGSI_PROCESSOR_VERTEX && next_pos_base == 60) { + /* add fake position export */ + if (ctx.type == TGSI_PROCESSOR_VERTEX && pos_emitted == false) { memset(&output[j], 0, sizeof(struct r600_bytecode_output)); output[j].gpr = 0; output[j].elem_size = 3; @@ -1485,15 +2038,14 @@ static int r600_shader_from_tgsi(struct r600_screen *rscreen, output[j].swizzle_z = 7; output[j].swizzle_w = 7; output[j].burst_count = 1; - output[j].barrier = 1; output[j].type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_POS; - output[j].array_base = next_pos_base; + output[j].array_base = 60; output[j].op = CF_OP_EXPORT; j++; - } + } - /* add fake param output for vertex shader if no param is exported */ - if (ctx.type == TGSI_PROCESSOR_VERTEX && next_param_base == 0) { + /* add fake param output for vertex shader if no param is exported */ + if (ctx.type == TGSI_PROCESSOR_VERTEX && next_param_base == 0) { memset(&output[j], 0, sizeof(struct r600_bytecode_output)); output[j].gpr = 0; output[j].elem_size = 3; @@ -1502,55 +2054,64 @@ static int r600_shader_from_tgsi(struct r600_screen *rscreen, output[j].swizzle_z = 7; output[j].swizzle_w = 7; output[j].burst_count = 1; - output[j].barrier = 1; output[j].type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_PARAM; output[j].array_base = 0; output[j].op = CF_OP_EXPORT; j++; - } + } + + /* add fake pixel export */ + if (ctx.type == TGSI_PROCESSOR_FRAGMENT && shader->nr_ps_color_exports == 0) { + memset(&output[j], 0, sizeof(struct r600_bytecode_output)); + output[j].gpr = 0; + output[j].elem_size = 3; + output[j].swizzle_x = 7; + output[j].swizzle_y = 7; + output[j].swizzle_z = 7; + output[j].swizzle_w = 7; + output[j].burst_count = 1; + output[j].type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_PIXEL; + output[j].array_base = 0; + output[j].op = CF_OP_EXPORT; + j++; + } + + noutput = j; - /* add fake pixel export */ - if (ctx.type == TGSI_PROCESSOR_FRAGMENT && shader->nr_ps_color_exports == 0) { - memset(&output[j], 0, sizeof(struct r600_bytecode_output)); - output[j].gpr = 0; - output[j].elem_size = 3; - output[j].swizzle_x = 7; - output[j].swizzle_y = 7; - output[j].swizzle_z = 7; - output[j].swizzle_w = 7; - output[j].burst_count = 1; - output[j].barrier = 1; - output[j].type = V_SQ_CF_ALLOC_EXPORT_WORD0_SQ_EXPORT_PIXEL; - output[j].array_base = 0; - output[j].op = CF_OP_EXPORT; - j++; - } - - noutput = j; - - /* set export done on last export of each type */ - for (i = noutput - 1, output_done = 0; i >= 0; i--) { - if (ctx.bc->chip_class < CAYMAN) { - if (i == (noutput - 1)) { - output[i].end_of_program = 1; + /* set export done on last export of each type */ + for (i = noutput - 1, output_done = 0; i >= 0; i--) { + if (!(output_done & (1 << output[i].type))) { + output_done |= (1 << output[i].type); + output[i].op = CF_OP_EXPORT_DONE; } } - if (!(output_done & (1 << output[i].type))) { - output_done |= (1 << output[i].type); - output[i].op = CF_OP_EXPORT_DONE; + /* add output to bytecode */ + if (!use_llvm) { + for (i = 0; i < noutput; i++) { + r = r600_bytecode_add_output(ctx.bc, &output[i]); + if (r) + goto out_err; + } } } - /* add output to bytecode */ + + /* add program end */ if (!use_llvm) { - for (i = 0; i < noutput; i++) { - r = r600_bytecode_add_output(ctx.bc, &output[i]); - if (r) - goto out_err; + if (ctx.bc->chip_class == CAYMAN) + cm_bytecode_add_cf_end(ctx.bc); + else { + const struct cf_op_info *last = NULL; + + if (ctx.bc->cf_last) + last = r600_isa_cf(ctx.bc->cf_last->op); + + /* alu clause instructions don't have EOP bit, so add NOP */ + if (!last || last->flags & CF_ALU || ctx.bc->cf_last->op == CF_OP_LOOP_END || ctx.bc->cf_last->op == CF_OP_CALL_FS) + r600_bytecode_add_cfinst(ctx.bc, CF_OP_NOP); + + ctx.bc->cf_last->end_of_program = 1; } } - /* add program end */ - if (!use_llvm && ctx.bc->chip_class == CAYMAN) - cm_bytecode_add_cf_end(ctx.bc); /* check GPR limit - we have 124 = 128 - 4 * (4 are reserved as alu clause temporary registers) */ @@ -1560,6 +2121,11 @@ static int r600_shader_from_tgsi(struct r600_screen *rscreen, goto out_err; } + if (ctx.type == TGSI_PROCESSOR_GEOMETRY) { + if ((r = generate_gs_copy_shader(rctx, pipeshader, &so))) + return r; + } + free(ctx.literals); tgsi_parse_free(&ctx.parse); return 0; @@ -5554,6 +6120,14 @@ static int tgsi_loop_brk_cont(struct r600_shader_ctx *ctx) return 0; } +static int tgsi_gs_emit(struct r600_shader_ctx *ctx) +{ + if (ctx->inst_info->op == CF_OP_EMIT_VERTEX) + emit_gs_ring_writes(ctx, TRUE); + + return r600_bytecode_add_cfinst(ctx->bc, ctx->inst_info->op); +} + static int tgsi_umad(struct r600_shader_ctx *ctx) { struct tgsi_full_instruction *inst = &ctx->parse.FullToken.FullInstruction; @@ -5735,8 +6309,8 @@ static struct r600_shader_tgsi_instruction r600_shader_tgsi_instruction[] = { {TGSI_OPCODE_TXF, 0, FETCH_OP_LD, tgsi_tex}, {TGSI_OPCODE_TXQ, 0, FETCH_OP_GET_TEXTURE_RESINFO, tgsi_tex}, {TGSI_OPCODE_CONT, 0, CF_OP_LOOP_CONTINUE, tgsi_loop_brk_cont}, - {TGSI_OPCODE_EMIT, 0, ALU_OP0_NOP, tgsi_unsupported}, - {TGSI_OPCODE_ENDPRIM, 0, ALU_OP0_NOP, tgsi_unsupported}, + {TGSI_OPCODE_EMIT, 0, CF_OP_EMIT_VERTEX, tgsi_gs_emit}, + {TGSI_OPCODE_ENDPRIM, 0, CF_OP_CUT_VERTEX, tgsi_gs_emit}, {TGSI_OPCODE_BGNLOOP, 0, ALU_OP0_NOP, tgsi_bgnloop}, {TGSI_OPCODE_BGNSUB, 0, ALU_OP0_NOP, tgsi_unsupported}, {TGSI_OPCODE_ENDLOOP, 0, ALU_OP0_NOP, tgsi_endloop}, @@ -5927,8 +6501,8 @@ static struct r600_shader_tgsi_instruction eg_shader_tgsi_instruction[] = { {TGSI_OPCODE_TXF, 0, FETCH_OP_LD, tgsi_tex}, {TGSI_OPCODE_TXQ, 0, FETCH_OP_GET_TEXTURE_RESINFO, tgsi_tex}, {TGSI_OPCODE_CONT, 0, CF_OP_LOOP_CONTINUE, tgsi_loop_brk_cont}, - {TGSI_OPCODE_EMIT, 0, ALU_OP0_NOP, tgsi_unsupported}, - {TGSI_OPCODE_ENDPRIM, 0, ALU_OP0_NOP, tgsi_unsupported}, + {TGSI_OPCODE_EMIT, 0, CF_OP_EMIT_VERTEX, tgsi_gs_emit}, + {TGSI_OPCODE_ENDPRIM, 0, CF_OP_CUT_VERTEX, tgsi_gs_emit}, {TGSI_OPCODE_BGNLOOP, 0, ALU_OP0_NOP, tgsi_bgnloop}, {TGSI_OPCODE_BGNSUB, 0, ALU_OP0_NOP, tgsi_unsupported}, {TGSI_OPCODE_ENDLOOP, 0, ALU_OP0_NOP, tgsi_endloop}, @@ -6119,8 +6693,8 @@ static struct r600_shader_tgsi_instruction cm_shader_tgsi_instruction[] = { {TGSI_OPCODE_TXF, 0, FETCH_OP_LD, tgsi_tex}, {TGSI_OPCODE_TXQ, 0, FETCH_OP_GET_TEXTURE_RESINFO, tgsi_tex}, {TGSI_OPCODE_CONT, 0, CF_OP_LOOP_CONTINUE, tgsi_loop_brk_cont}, - {TGSI_OPCODE_EMIT, 0, ALU_OP0_NOP, tgsi_unsupported}, - {TGSI_OPCODE_ENDPRIM, 0, ALU_OP0_NOP, tgsi_unsupported}, + {TGSI_OPCODE_EMIT, 0, CF_OP_EMIT_VERTEX, tgsi_gs_emit}, + {TGSI_OPCODE_ENDPRIM, 0, CF_OP_CUT_VERTEX, tgsi_gs_emit}, {TGSI_OPCODE_BGNLOOP, 0, ALU_OP0_NOP, tgsi_bgnloop}, {TGSI_OPCODE_BGNSUB, 0, ALU_OP0_NOP, tgsi_unsupported}, {TGSI_OPCODE_ENDLOOP, 0, ALU_OP0_NOP, tgsi_endloop}, diff --git a/src/gallium/drivers/r600/r600_shader.h b/src/gallium/drivers/r600/r600_shader.h index d989ce43649..89602c7013f 100644 --- a/src/gallium/drivers/r600/r600_shader.h +++ b/src/gallium/drivers/r600/r600_shader.h @@ -37,6 +37,7 @@ struct r600_shader_io { unsigned lds_pos; /* for evergreen */ unsigned back_color_input; unsigned write_mask; + int ring_offset; }; struct r600_shader { @@ -61,12 +62,22 @@ struct r600_shader { /* flag is set if the shader writes VS_OUT_MISC_VEC (e.g. for PSIZE) */ boolean vs_out_misc_write; boolean vs_out_point_size; + boolean vs_out_layer; boolean has_txq_cube_array_z_comp; boolean uses_tex_buffers; + boolean gs_prim_id_input; + + /* geometry shader properties */ + unsigned gs_input_prim; + unsigned gs_output_prim; + unsigned gs_max_out_vertices; + /* size in bytes of a data item in the ring (single vertex data) */ + unsigned ring_item_size; unsigned indirect_files; unsigned max_arrays; unsigned num_arrays; + unsigned vs_as_es; struct r600_shader_array * arrays; }; @@ -74,6 +85,7 @@ struct r600_shader_key { unsigned color_two_side:1; unsigned alpha_to_one:1; unsigned nr_cbufs:4; + unsigned vs_as_es:1; }; struct r600_shader_array { @@ -85,6 +97,8 @@ struct r600_shader_array { struct r600_pipe_shader { struct r600_pipe_shader_selector *selector; struct r600_pipe_shader *next_variant; + /* for GS - corresponding copy shader (installed as VS) */ + struct r600_pipe_shader *gs_copy_shader; struct r600_shader shader; struct r600_command_buffer command_buffer; /* register writes */ struct r600_resource *bo; diff --git a/src/gallium/drivers/r600/r600_state.c b/src/gallium/drivers/r600/r600_state.c index a0d527bcfee..1b73ce30213 100644 --- a/src/gallium/drivers/r600/r600_state.c +++ b/src/gallium/drivers/r600/r600_state.c @@ -1264,6 +1264,7 @@ static void r600_init_color_surface(struct r600_context *rctx, unsigned level = surf->base.u.tex.level; unsigned pitch, slice; unsigned color_info; + unsigned color_view; unsigned format, swap, ntype, endian; unsigned offset; const struct util_format_description *desc; @@ -1277,10 +1278,15 @@ static void r600_init_color_surface(struct r600_context *rctx, } offset = rtex->surface.level[level].offset; - if (rtex->surface.level[level].mode < RADEON_SURF_MODE_1D) { + if (rtex->surface.level[level].mode == RADEON_SURF_MODE_LINEAR) { + assert(surf->base.u.tex.first_layer == surf->base.u.tex.last_layer); offset += rtex->surface.level[level].slice_size * - surf->base.u.tex.first_layer; - } + surf->base.u.tex.first_layer; + color_view = 0; + } else + color_view = S_028080_SLICE_START(surf->base.u.tex.first_layer) | + S_028080_SLICE_MAX(surf->base.u.tex.last_layer); + pitch = rtex->surface.level[level].nblk_x / 8 - 1; slice = (rtex->surface.level[level].nblk_x * rtex->surface.level[level].nblk_y) / 64; if (slice) { @@ -1466,14 +1472,7 @@ static void r600_init_color_surface(struct r600_context *rctx, } surf->cb_color_info = color_info; - - if (rtex->surface.level[level].mode < RADEON_SURF_MODE_1D) { - surf->cb_color_view = 0; - } else { - surf->cb_color_view = S_028080_SLICE_START(surf->base.u.tex.first_layer) | - S_028080_SLICE_MAX(surf->base.u.tex.last_layer); - } - + surf->cb_color_view = color_view; surf->color_initialized = true; } @@ -1667,8 +1666,6 @@ static void r600_set_framebuffer_state(struct pipe_context *ctx, rctx->alphatest_state.atom.dirty = true; } - r600_update_db_shader_control(rctx); - /* Calculate the CS size. */ rctx->framebuffer.atom.num_dw = 10 /*COLOR_INFO*/ + 4 /*SCISSOR*/ + 3 /*SHADER_CONTROL*/ + 8 /*MSAA*/; @@ -2067,6 +2064,7 @@ static void r600_emit_config_state(struct r600_context *rctx, struct r600_atom * struct r600_config_state *a = (struct r600_config_state*)atom; r600_write_config_reg(cs, R_008C04_SQ_GPR_RESOURCE_MGMT_1, a->sq_gpr_resource_mgmt_1); + r600_write_config_reg(cs, R_008C08_SQ_GPR_RESOURCE_MGMT_2, a->sq_gpr_resource_mgmt_2); } static void r600_emit_vertex_buffers(struct r600_context *rctx, struct r600_atom *atom) @@ -2118,16 +2116,18 @@ static void r600_emit_constant_buffers(struct r600_context *rctx, struct r600_resource *rbuffer; unsigned offset; unsigned buffer_index = ffs(dirty_mask) - 1; - + unsigned gs_ring_buffer = (buffer_index == R600_GS_RING_CONST_BUFFER); cb = &state->cb[buffer_index]; rbuffer = (struct r600_resource*)cb->buffer; assert(rbuffer); offset = cb->buffer_offset; - r600_write_context_reg(cs, reg_alu_constbuf_size + buffer_index * 4, - ALIGN_DIVUP(cb->buffer_size >> 4, 16)); - r600_write_context_reg(cs, reg_alu_const_cache + buffer_index * 4, offset >> 8); + if (!gs_ring_buffer) { + r600_write_context_reg(cs, reg_alu_constbuf_size + buffer_index * 4, + ALIGN_DIVUP(cb->buffer_size >> 4, 16)); + r600_write_context_reg(cs, reg_alu_const_cache + buffer_index * 4, offset >> 8); + } radeon_emit(cs, PKT3(PKT3_NOP, 0, 0)); radeon_emit(cs, r600_context_bo_reloc(&rctx->b, &rctx->b.rings.gfx, rbuffer, RADEON_USAGE_READ)); @@ -2137,8 +2137,8 @@ static void r600_emit_constant_buffers(struct r600_context *rctx, radeon_emit(cs, offset); /* RESOURCEi_WORD0 */ radeon_emit(cs, rbuffer->buf->size - offset - 1); /* RESOURCEi_WORD1 */ radeon_emit(cs, /* RESOURCEi_WORD2 */ - S_038008_ENDIAN_SWAP(r600_endian_swap(32)) | - S_038008_STRIDE(16)); + S_038008_ENDIAN_SWAP(gs_ring_buffer ? ENDIAN_NONE : r600_endian_swap(32)) | + S_038008_STRIDE(gs_ring_buffer ? 4 : 16)); radeon_emit(cs, 0); /* RESOURCEi_WORD3 */ radeon_emit(cs, 0); /* RESOURCEi_WORD4 */ radeon_emit(cs, 0); /* RESOURCEi_WORD5 */ @@ -2323,34 +2323,124 @@ static void r600_emit_vertex_fetch_shader(struct r600_context *rctx, struct r600 radeon_emit(cs, r600_context_bo_reloc(&rctx->b, &rctx->b.rings.gfx, shader->buffer, RADEON_USAGE_READ)); } +static void r600_emit_shader_stages(struct r600_context *rctx, struct r600_atom *a) +{ + struct radeon_winsys_cs *cs = rctx->b.rings.gfx.cs; + struct r600_shader_stages_state *state = (struct r600_shader_stages_state*)a; + + uint32_t v2 = 0, primid = 0; + + if (state->geom_enable) { + uint32_t cut_val; + + if (rctx->gs_shader->current->shader.gs_max_out_vertices <= 128) + cut_val = V_028A40_GS_CUT_128; + else if (rctx->gs_shader->current->shader.gs_max_out_vertices <= 256) + cut_val = V_028A40_GS_CUT_256; + else if (rctx->gs_shader->current->shader.gs_max_out_vertices <= 512) + cut_val = V_028A40_GS_CUT_512; + else + cut_val = V_028A40_GS_CUT_1024; + + v2 = S_028A40_MODE(V_028A40_GS_SCENARIO_G) | + S_028A40_CUT_MODE(cut_val); + + if (rctx->gs_shader->current->shader.gs_prim_id_input) + primid = 1; + } + + r600_write_context_reg(cs, R_028A40_VGT_GS_MODE, v2); + r600_write_context_reg(cs, R_028A84_VGT_PRIMITIVEID_EN, primid); +} + +static void r600_emit_gs_rings(struct r600_context *rctx, struct r600_atom *a) +{ + struct pipe_screen *screen = rctx->b.b.screen; + struct radeon_winsys_cs *cs = rctx->b.rings.gfx.cs; + struct r600_gs_rings_state *state = (struct r600_gs_rings_state*)a; + struct r600_resource *rbuffer; + + r600_write_config_reg(cs, R_008040_WAIT_UNTIL, S_008040_WAIT_3D_IDLE(1)); + radeon_emit(cs, PKT3(PKT3_EVENT_WRITE, 0, 0)); + radeon_emit(cs, EVENT_TYPE(EVENT_TYPE_VGT_FLUSH)); + + if (state->enable) { + rbuffer =(struct r600_resource*)state->esgs_ring.buffer; + r600_write_config_reg(cs, R_008C40_SQ_ESGS_RING_BASE, + (r600_resource_va(screen, &rbuffer->b.b)) >> 8); + radeon_emit(cs, PKT3(PKT3_NOP, 0, 0)); + radeon_emit(cs, r600_context_bo_reloc(&rctx->b, &rctx->b.rings.gfx, rbuffer, RADEON_USAGE_READWRITE)); + r600_write_config_reg(cs, R_008C44_SQ_ESGS_RING_SIZE, + state->esgs_ring.buffer_size >> 8); + + rbuffer =(struct r600_resource*)state->gsvs_ring.buffer; + r600_write_config_reg(cs, R_008C48_SQ_GSVS_RING_BASE, + (r600_resource_va(screen, &rbuffer->b.b)) >> 8); + radeon_emit(cs, PKT3(PKT3_NOP, 0, 0)); + radeon_emit(cs, r600_context_bo_reloc(&rctx->b, &rctx->b.rings.gfx, rbuffer, RADEON_USAGE_READWRITE)); + r600_write_config_reg(cs, R_008C4C_SQ_GSVS_RING_SIZE, + state->gsvs_ring.buffer_size >> 8); + } else { + r600_write_config_reg(cs, R_008C44_SQ_ESGS_RING_SIZE, 0); + r600_write_config_reg(cs, R_008C4C_SQ_GSVS_RING_SIZE, 0); + } + + r600_write_config_reg(cs, R_008040_WAIT_UNTIL, S_008040_WAIT_3D_IDLE(1)); + radeon_emit(cs, PKT3(PKT3_EVENT_WRITE, 0, 0)); + radeon_emit(cs, EVENT_TYPE(EVENT_TYPE_VGT_FLUSH)); +} + /* Adjust GPR allocation on R6xx/R7xx */ bool r600_adjust_gprs(struct r600_context *rctx) { unsigned num_ps_gprs = rctx->ps_shader->current->shader.bc.ngpr; - unsigned num_vs_gprs = rctx->vs_shader->current->shader.bc.ngpr; + unsigned num_vs_gprs, num_es_gprs, num_gs_gprs; unsigned new_num_ps_gprs = num_ps_gprs; - unsigned new_num_vs_gprs = num_vs_gprs; + unsigned new_num_vs_gprs, new_num_es_gprs, new_num_gs_gprs; unsigned cur_num_ps_gprs = G_008C04_NUM_PS_GPRS(rctx->config_state.sq_gpr_resource_mgmt_1); unsigned cur_num_vs_gprs = G_008C04_NUM_VS_GPRS(rctx->config_state.sq_gpr_resource_mgmt_1); + unsigned cur_num_gs_gprs = G_008C08_NUM_GS_GPRS(rctx->config_state.sq_gpr_resource_mgmt_2); + unsigned cur_num_es_gprs = G_008C08_NUM_ES_GPRS(rctx->config_state.sq_gpr_resource_mgmt_2); unsigned def_num_ps_gprs = rctx->default_ps_gprs; unsigned def_num_vs_gprs = rctx->default_vs_gprs; + unsigned def_num_gs_gprs = 0; + unsigned def_num_es_gprs = 0; unsigned def_num_clause_temp_gprs = rctx->r6xx_num_clause_temp_gprs; /* hardware will reserve twice num_clause_temp_gprs */ - unsigned max_gprs = def_num_ps_gprs + def_num_vs_gprs + def_num_clause_temp_gprs * 2; - unsigned tmp; + unsigned max_gprs = def_num_gs_gprs + def_num_es_gprs + def_num_ps_gprs + def_num_vs_gprs + def_num_clause_temp_gprs * 2; + unsigned tmp, tmp2; + + if (rctx->gs_shader) { + num_es_gprs = rctx->vs_shader->current->shader.bc.ngpr; + num_gs_gprs = rctx->gs_shader->current->shader.bc.ngpr; + num_vs_gprs = rctx->gs_shader->current->gs_copy_shader->shader.bc.ngpr; + } else { + num_es_gprs = 0; + num_gs_gprs = 0; + num_vs_gprs = rctx->vs_shader->current->shader.bc.ngpr; + } + new_num_vs_gprs = num_vs_gprs; + new_num_es_gprs = num_es_gprs; + new_num_gs_gprs = num_gs_gprs; /* the sum of all SQ_GPR_RESOURCE_MGMT*.NUM_*_GPRS must <= to max_gprs */ - if (new_num_ps_gprs > cur_num_ps_gprs || new_num_vs_gprs > cur_num_vs_gprs) { + if (new_num_ps_gprs > cur_num_ps_gprs || new_num_vs_gprs > cur_num_vs_gprs || + new_num_es_gprs > cur_num_es_gprs || new_num_gs_gprs > cur_num_gs_gprs) { /* try to use switch back to default */ - if (new_num_ps_gprs > def_num_ps_gprs || new_num_vs_gprs > def_num_vs_gprs) { + if (new_num_ps_gprs > def_num_ps_gprs || new_num_vs_gprs > def_num_vs_gprs || + new_num_gs_gprs > def_num_gs_gprs || new_num_es_gprs > def_num_es_gprs) { /* always privilege vs stage so that at worst we have the * pixel stage producing wrong output (not the vertex * stage) */ - new_num_ps_gprs = max_gprs - (new_num_vs_gprs + def_num_clause_temp_gprs * 2); + new_num_ps_gprs = max_gprs - ((new_num_vs_gprs - new_num_es_gprs - new_num_gs_gprs) + def_num_clause_temp_gprs * 2); new_num_vs_gprs = num_vs_gprs; + new_num_gs_gprs = num_gs_gprs; + new_num_es_gprs = num_es_gprs; } else { new_num_ps_gprs = def_num_ps_gprs; new_num_vs_gprs = def_num_vs_gprs; + new_num_es_gprs = def_num_es_gprs; + new_num_gs_gprs = def_num_gs_gprs; } } else { return true; @@ -2362,10 +2452,11 @@ bool r600_adjust_gprs(struct r600_context *rctx) * it will lockup. So in this case just discard the draw command * and don't change the current gprs repartitions. */ - if (num_ps_gprs > new_num_ps_gprs || num_vs_gprs > new_num_vs_gprs) { - R600_ERR("ps & vs shader require too many register (%d + %d) " + if (num_ps_gprs > new_num_ps_gprs || num_vs_gprs > new_num_vs_gprs || + num_gs_gprs > new_num_gs_gprs || num_es_gprs > new_num_es_gprs) { + R600_ERR("shaders require too many register (%d + %d + %d + %d) " "for a combined maximum of %d\n", - num_ps_gprs, num_vs_gprs, max_gprs); + num_ps_gprs, num_vs_gprs, num_es_gprs, num_gs_gprs, max_gprs); return false; } @@ -2373,8 +2464,12 @@ bool r600_adjust_gprs(struct r600_context *rctx) tmp = S_008C04_NUM_PS_GPRS(new_num_ps_gprs) | S_008C04_NUM_VS_GPRS(new_num_vs_gprs) | S_008C04_NUM_CLAUSE_TEMP_GPRS(def_num_clause_temp_gprs); - if (rctx->config_state.sq_gpr_resource_mgmt_1 != tmp) { + + tmp2 = S_008C08_NUM_ES_GPRS(new_num_es_gprs) | + S_008C08_NUM_GS_GPRS(new_num_gs_gprs); + if (rctx->config_state.sq_gpr_resource_mgmt_1 != tmp || rctx->config_state.sq_gpr_resource_mgmt_2 != tmp2) { rctx->config_state.sq_gpr_resource_mgmt_1 = tmp; + rctx->config_state.sq_gpr_resource_mgmt_2 = tmp2; rctx->config_state.atom.dirty = true; rctx->b.flags |= R600_CONTEXT_WAIT_3D_IDLE; } @@ -2492,19 +2587,19 @@ void r600_init_atom_start_cs(struct r600_context *rctx) num_es_stack_entries = 16; break; case CHIP_RV770: - num_ps_gprs = 192; + num_ps_gprs = 130; num_vs_gprs = 56; num_temp_gprs = 4; - num_gs_gprs = 0; - num_es_gprs = 0; - num_ps_threads = 188; + num_gs_gprs = 31; + num_es_gprs = 31; + num_ps_threads = 180; num_vs_threads = 60; - num_gs_threads = 0; - num_es_threads = 0; - num_ps_stack_entries = 256; - num_vs_stack_entries = 256; - num_gs_stack_entries = 0; - num_es_stack_entries = 0; + num_gs_threads = 4; + num_es_threads = 4; + num_ps_stack_entries = 128; + num_vs_stack_entries = 128; + num_gs_stack_entries = 128; + num_es_stack_entries = 128; break; case CHIP_RV730: case CHIP_RV740: @@ -2513,10 +2608,10 @@ void r600_init_atom_start_cs(struct r600_context *rctx) num_temp_gprs = 4; num_gs_gprs = 0; num_es_gprs = 0; - num_ps_threads = 188; + num_ps_threads = 180; num_vs_threads = 60; - num_gs_threads = 0; - num_es_threads = 0; + num_gs_threads = 4; + num_es_threads = 4; num_ps_stack_entries = 128; num_vs_stack_entries = 128; num_gs_stack_entries = 0; @@ -2528,10 +2623,10 @@ void r600_init_atom_start_cs(struct r600_context *rctx) num_temp_gprs = 4; num_gs_gprs = 0; num_es_gprs = 0; - num_ps_threads = 144; + num_ps_threads = 136; num_vs_threads = 48; - num_gs_threads = 0; - num_es_threads = 0; + num_gs_threads = 4; + num_es_threads = 4; num_ps_stack_entries = 128; num_vs_stack_entries = 128; num_gs_stack_entries = 0; @@ -2707,9 +2802,12 @@ void r600_init_atom_start_cs(struct r600_context *rctx) r600_store_value(cb, 0); /* R_028240_PA_SC_GENERIC_SCISSOR_TL */ r600_store_value(cb, S_028244_BR_X(8192) | S_028244_BR_Y(8192)); /* R_028244_PA_SC_GENERIC_SCISSOR_BR */ - r600_store_context_reg_seq(cb, R_0288CC_SQ_PGM_CF_OFFSET_PS, 2); + r600_store_context_reg_seq(cb, R_0288CC_SQ_PGM_CF_OFFSET_PS, 5); r600_store_value(cb, 0); /* R_0288CC_SQ_PGM_CF_OFFSET_PS */ r600_store_value(cb, 0); /* R_0288D0_SQ_PGM_CF_OFFSET_VS */ + r600_store_value(cb, 0); /* R_0288D4_SQ_PGM_CF_OFFSET_GS */ + r600_store_value(cb, 0); /* R_0288D8_SQ_PGM_CF_OFFSET_ES */ + r600_store_value(cb, 0); /* R_0288DC_SQ_PGM_CF_OFFSET_FS */ r600_store_context_reg(cb, R_0288E0_SQ_VTX_SEMANTIC_CLEAR, ~0); @@ -2718,7 +2816,6 @@ void r600_init_atom_start_cs(struct r600_context *rctx) r600_store_value(cb, 0); /* R_028404_VGT_MIN_VTX_INDX */ r600_store_context_reg(cb, R_0288A4_SQ_PGM_RESOURCES_FS, 0); - r600_store_context_reg(cb, R_0288DC_SQ_PGM_CF_OFFSET_FS, 0); if (rctx->b.chip_class == R700 && rctx->screen->b.has_streamout) r600_store_context_reg(cb, R_028354_SX_SURFACE_SYNC, S_028354_SURFACE_SYNC_MASK(0xf)); @@ -2729,6 +2826,7 @@ void r600_init_atom_start_cs(struct r600_context *rctx) r600_store_loop_const(cb, R_03E200_SQ_LOOP_CONST_0, 0x1000FFF); r600_store_loop_const(cb, R_03E200_SQ_LOOP_CONST_0 + (32 * 4), 0x1000FFF); + r600_store_loop_const(cb, R_03E200_SQ_LOOP_CONST_0 + (64 * 4), 0x1000FFF); } void r600_update_ps_state(struct pipe_context *ctx, struct r600_pipe_shader *shader) @@ -2901,6 +2999,94 @@ void r600_update_vs_state(struct pipe_context *ctx, struct r600_pipe_shader *sha S_02881C_USE_VTX_POINT_SIZE(rshader->vs_out_point_size); } +static unsigned r600_conv_prim_to_gs_out(unsigned mode) +{ + static const int prim_conv[] = { + V_028A6C_OUTPRIM_TYPE_POINTLIST, + V_028A6C_OUTPRIM_TYPE_LINESTRIP, + V_028A6C_OUTPRIM_TYPE_LINESTRIP, + V_028A6C_OUTPRIM_TYPE_LINESTRIP, + V_028A6C_OUTPRIM_TYPE_TRISTRIP, + V_028A6C_OUTPRIM_TYPE_TRISTRIP, + V_028A6C_OUTPRIM_TYPE_TRISTRIP, + V_028A6C_OUTPRIM_TYPE_TRISTRIP, + V_028A6C_OUTPRIM_TYPE_TRISTRIP, + V_028A6C_OUTPRIM_TYPE_TRISTRIP, + V_028A6C_OUTPRIM_TYPE_LINESTRIP, + V_028A6C_OUTPRIM_TYPE_LINESTRIP, + V_028A6C_OUTPRIM_TYPE_TRISTRIP, + V_028A6C_OUTPRIM_TYPE_TRISTRIP, + V_028A6C_OUTPRIM_TYPE_TRISTRIP + }; + assert(mode < Elements(prim_conv)); + + return prim_conv[mode]; +} + +void r600_update_gs_state(struct pipe_context *ctx, struct r600_pipe_shader *shader) +{ + struct r600_context *rctx = (struct r600_context *)ctx; + struct r600_command_buffer *cb = &shader->command_buffer; + struct r600_shader *rshader = &shader->shader; + struct r600_shader *cp_shader = &shader->gs_copy_shader->shader; + unsigned gsvs_itemsize = + (cp_shader->ring_item_size * rshader->gs_max_out_vertices) >> 2; + + r600_init_command_buffer(cb, 64); + + /* VGT_GS_MODE is written by r600_emit_shader_stages */ + r600_store_context_reg(cb, R_028AB8_VGT_VTX_CNT_EN, 1); + + if (rctx->b.chip_class >= R700) { + r600_store_context_reg(cb, R_028B38_VGT_GS_MAX_VERT_OUT, + S_028B38_MAX_VERT_OUT(rshader->gs_max_out_vertices)); + } + r600_store_context_reg(cb, R_028A6C_VGT_GS_OUT_PRIM_TYPE, + r600_conv_prim_to_gs_out(rshader->gs_output_prim)); + + r600_store_context_reg_seq(cb, R_0288C8_SQ_GS_VERT_ITEMSIZE, 4); + r600_store_value(cb, cp_shader->ring_item_size >> 2); + r600_store_value(cb, 0); + r600_store_value(cb, 0); + r600_store_value(cb, 0); + + r600_store_context_reg(cb, R_0288A8_SQ_ESGS_RING_ITEMSIZE, + (rshader->ring_item_size) >> 2); + + r600_store_context_reg(cb, R_0288AC_SQ_GSVS_RING_ITEMSIZE, + gsvs_itemsize); + + /* FIXME calculate these values somehow ??? */ + r600_store_config_reg_seq(cb, R_0088C8_VGT_GS_PER_ES, 2); + r600_store_value(cb, 0x80); /* GS_PER_ES */ + r600_store_value(cb, 0x100); /* ES_PER_GS */ + r600_store_config_reg_seq(cb, R_0088E8_VGT_GS_PER_VS, 1); + r600_store_value(cb, 0x2); /* GS_PER_VS */ + + r600_store_context_reg(cb, R_02887C_SQ_PGM_RESOURCES_GS, + S_02887C_NUM_GPRS(rshader->bc.ngpr) | + S_02887C_STACK_SIZE(rshader->bc.nstack)); + r600_store_context_reg(cb, R_02886C_SQ_PGM_START_GS, + r600_resource_va(ctx->screen, (void *)shader->bo) >> 8); + /* After that, the NOP relocation packet must be emitted (shader->bo, RADEON_USAGE_READ). */ +} + +void r600_update_es_state(struct pipe_context *ctx, struct r600_pipe_shader *shader) +{ + struct r600_command_buffer *cb = &shader->command_buffer; + struct r600_shader *rshader = &shader->shader; + + r600_init_command_buffer(cb, 32); + + r600_store_context_reg(cb, R_028890_SQ_PGM_RESOURCES_ES, + S_028890_NUM_GPRS(rshader->bc.ngpr) | + S_028890_STACK_SIZE(rshader->bc.nstack)); + r600_store_context_reg(cb, R_028880_SQ_PGM_START_ES, + r600_resource_va(ctx->screen, (void *)shader->bo) >> 8); + /* After that, the NOP relocation packet must be emitted (shader->bo, RADEON_USAGE_READ). */ +} + + void *r600_create_resolve_blend(struct r600_context *rctx) { struct pipe_blend_state blend; @@ -3262,6 +3448,10 @@ void r600_init_state_functions(struct r600_context *rctx) rctx->atoms[id++] = &rctx->b.streamout.begin_atom; r600_init_atom(rctx, &rctx->vertex_shader.atom, id++, r600_emit_shader, 23); r600_init_atom(rctx, &rctx->pixel_shader.atom, id++, r600_emit_shader, 0); + r600_init_atom(rctx, &rctx->geometry_shader.atom, id++, r600_emit_shader, 0); + r600_init_atom(rctx, &rctx->export_shader.atom, id++, r600_emit_shader, 0); + r600_init_atom(rctx, &rctx->shader_stages.atom, id++, r600_emit_shader_stages, 0); + r600_init_atom(rctx, &rctx->gs_rings.atom, id++, r600_emit_gs_rings, 0); rctx->b.b.create_blend_state = r600_create_blend_state; rctx->b.b.create_depth_stencil_alpha_state = r600_create_dsa_state; diff --git a/src/gallium/drivers/r600/r600_state_common.c b/src/gallium/drivers/r600/r600_state_common.c index d8fab1083e1..d1410260cc5 100644 --- a/src/gallium/drivers/r600/r600_state_common.c +++ b/src/gallium/drivers/r600/r600_state_common.c @@ -301,11 +301,6 @@ static void r600_bind_dsa_state(struct pipe_context *ctx, void *state) rctx->alphatest_state.sx_alpha_test_control = dsa->sx_alpha_test_control; rctx->alphatest_state.sx_alpha_ref = dsa->alpha_ref; rctx->alphatest_state.atom.dirty = true; - if (rctx->b.chip_class >= EVERGREEN) { - evergreen_update_db_shader_control(rctx); - } else { - r600_update_db_shader_control(rctx); - } } } @@ -698,6 +693,8 @@ static INLINE struct r600_shader_key r600_shader_selector_key(struct pipe_contex /* Dual-source blending only makes sense with nr_cbufs == 1. */ if (key.nr_cbufs == 1 && rctx->dual_src_blend) key.nr_cbufs = 2; + } else if (sel->type == PIPE_SHADER_VERTEX) { + key.vs_as_es = (rctx->gs_shader != NULL); } return key; } @@ -709,7 +706,6 @@ static int r600_shader_select(struct pipe_context *ctx, bool *dirty) { struct r600_shader_key key; - struct r600_context *rctx = (struct r600_context *)ctx; struct r600_pipe_shader * shader = NULL; int r; @@ -771,11 +767,6 @@ static int r600_shader_select(struct pipe_context *ctx, shader->next_variant = sel->current; sel->current = shader; - if (rctx->ps_shader && - rctx->cb_misc_state.nr_ps_color_outputs != rctx->ps_shader->current->nr_ps_color_outputs) { - rctx->cb_misc_state.nr_ps_color_outputs = rctx->ps_shader->current->nr_ps_color_outputs; - rctx->cb_misc_state.atom.dirty = true; - } return 0; } @@ -784,16 +775,10 @@ static void *r600_create_shader_state(struct pipe_context *ctx, unsigned pipe_shader_type) { struct r600_pipe_shader_selector *sel = CALLOC_STRUCT(r600_pipe_shader_selector); - int r; sel->type = pipe_shader_type; sel->tokens = tgsi_dup_tokens(state->tokens); sel->so = state->stream_output; - - r = r600_shader_select(ctx, sel, NULL); - if (r) - return NULL; - return sel; } @@ -809,6 +794,12 @@ static void *r600_create_vs_state(struct pipe_context *ctx, return r600_create_shader_state(ctx, state, PIPE_SHADER_VERTEX); } +static void *r600_create_gs_state(struct pipe_context *ctx, + const struct pipe_shader_state *state) +{ + return r600_create_shader_state(ctx, state, PIPE_SHADER_GEOMETRY); +} + static void r600_bind_ps_state(struct pipe_context *ctx, void *state) { struct r600_context *rctx = (struct r600_context *)ctx; @@ -816,31 +807,7 @@ static void r600_bind_ps_state(struct pipe_context *ctx, void *state) if (!state) state = rctx->dummy_pixel_shader; - rctx->pixel_shader.shader = rctx->ps_shader = (struct r600_pipe_shader_selector *)state; - rctx->pixel_shader.atom.num_dw = rctx->ps_shader->current->command_buffer.num_dw; - rctx->pixel_shader.atom.dirty = true; - - r600_context_add_resource_size(ctx, (struct pipe_resource *)rctx->ps_shader->current->bo); - - if (rctx->b.chip_class <= R700) { - bool multiwrite = rctx->ps_shader->current->shader.fs_write_all; - - if (rctx->cb_misc_state.multiwrite != multiwrite) { - rctx->cb_misc_state.multiwrite = multiwrite; - rctx->cb_misc_state.atom.dirty = true; - } - } - - if (rctx->cb_misc_state.nr_ps_color_outputs != rctx->ps_shader->current->nr_ps_color_outputs) { - rctx->cb_misc_state.nr_ps_color_outputs = rctx->ps_shader->current->nr_ps_color_outputs; - rctx->cb_misc_state.atom.dirty = true; - } - - if (rctx->b.chip_class >= EVERGREEN) { - evergreen_update_db_shader_control(rctx); - } else { - r600_update_db_shader_control(rctx); - } + rctx->ps_shader = (struct r600_pipe_shader_selector *)state; } static void r600_bind_vs_state(struct pipe_context *ctx, void *state) @@ -850,19 +817,19 @@ static void r600_bind_vs_state(struct pipe_context *ctx, void *state) if (!state) return; - rctx->vertex_shader.shader = rctx->vs_shader = (struct r600_pipe_shader_selector *)state; - rctx->vertex_shader.atom.dirty = true; + rctx->vs_shader = (struct r600_pipe_shader_selector *)state; rctx->b.streamout.stride_in_dw = rctx->vs_shader->so.stride; +} - r600_context_add_resource_size(ctx, (struct pipe_resource *)rctx->vs_shader->current->bo); +static void r600_bind_gs_state(struct pipe_context *ctx, void *state) +{ + struct r600_context *rctx = (struct r600_context *)ctx; - /* Update clip misc state. */ - if (rctx->vs_shader->current->pa_cl_vs_out_cntl != rctx->clip_misc_state.pa_cl_vs_out_cntl || - rctx->vs_shader->current->shader.clip_dist_write != rctx->clip_misc_state.clip_dist_write) { - rctx->clip_misc_state.pa_cl_vs_out_cntl = rctx->vs_shader->current->pa_cl_vs_out_cntl; - rctx->clip_misc_state.clip_dist_write = rctx->vs_shader->current->shader.clip_dist_write; - rctx->clip_misc_state.atom.dirty = true; - } + rctx->gs_shader = (struct r600_pipe_shader_selector *)state; + + if (!state) + return; + rctx->b.streamout.stride_in_dw = rctx->gs_shader->so.stride; } static void r600_delete_shader_selector(struct pipe_context *ctx, @@ -905,6 +872,20 @@ static void r600_delete_vs_state(struct pipe_context *ctx, void *state) r600_delete_shader_selector(ctx, sel); } + +static void r600_delete_gs_state(struct pipe_context *ctx, void *state) +{ + struct r600_context *rctx = (struct r600_context *)ctx; + struct r600_pipe_shader_selector *sel = (struct r600_pipe_shader_selector *)state; + + if (rctx->gs_shader == sel) { + rctx->gs_shader = NULL; + } + + r600_delete_shader_selector(ctx, sel); +} + + void r600_constant_buffers_dirty(struct r600_context *rctx, struct r600_constbuf_state *state) { if (state->dirty_mask) { @@ -1098,10 +1079,65 @@ static void r600_setup_txq_cube_array_constants(struct r600_context *rctx, int s pipe_resource_reference(&cb.buffer, NULL); } +static void update_shader_atom(struct pipe_context *ctx, + struct r600_shader_state *state, + struct r600_pipe_shader *shader) +{ + state->shader = shader; + if (shader) { + state->atom.num_dw = shader->command_buffer.num_dw; + state->atom.dirty = true; + r600_context_add_resource_size(ctx, (struct pipe_resource *)shader->bo); + } else { + state->atom.num_dw = 0; + state->atom.dirty = false; + } +} + +static void update_gs_block_state(struct r600_context *rctx, unsigned enable) +{ + if (rctx->shader_stages.geom_enable != enable) { + rctx->shader_stages.geom_enable = enable; + rctx->shader_stages.atom.dirty = true; + } + + if (rctx->gs_rings.enable != enable) { + rctx->gs_rings.enable = enable; + rctx->gs_rings.atom.dirty = true; + + if (enable && !rctx->gs_rings.esgs_ring.buffer) { + unsigned size = 0x1C000; + rctx->gs_rings.esgs_ring.buffer = + pipe_buffer_create(rctx->b.b.screen, PIPE_BIND_CUSTOM, + PIPE_USAGE_STATIC, size); + rctx->gs_rings.esgs_ring.buffer_size = size; + + size = 0x4000000; + + rctx->gs_rings.gsvs_ring.buffer = + pipe_buffer_create(rctx->b.b.screen, PIPE_BIND_CUSTOM, + PIPE_USAGE_STATIC, size); + rctx->gs_rings.gsvs_ring.buffer_size = size; + } + + if (enable) { + r600_set_constant_buffer(&rctx->b.b, PIPE_SHADER_GEOMETRY, + R600_GS_RING_CONST_BUFFER, &rctx->gs_rings.esgs_ring); + r600_set_constant_buffer(&rctx->b.b, PIPE_SHADER_VERTEX, + R600_GS_RING_CONST_BUFFER, &rctx->gs_rings.gsvs_ring); + } else { + r600_set_constant_buffer(&rctx->b.b, PIPE_SHADER_GEOMETRY, + R600_GS_RING_CONST_BUFFER, NULL); + r600_set_constant_buffer(&rctx->b.b, PIPE_SHADER_VERTEX, + R600_GS_RING_CONST_BUFFER, NULL); + } + } +} + static bool r600_update_derived_state(struct r600_context *rctx) { struct pipe_context * ctx = (struct pipe_context*)rctx; - bool ps_dirty = false; + bool ps_dirty = false, vs_dirty = false, gs_dirty = false; bool blend_disable; if (!rctx->blitter->running) { @@ -1119,23 +1155,101 @@ static bool r600_update_derived_state(struct r600_context *rctx) } } - r600_shader_select(ctx, rctx->ps_shader, &ps_dirty); + update_gs_block_state(rctx, rctx->gs_shader != NULL); + + if (rctx->gs_shader) { + r600_shader_select(ctx, rctx->gs_shader, &gs_dirty); + if (unlikely(!rctx->gs_shader->current)) + return false; + + if (!rctx->shader_stages.geom_enable) { + rctx->shader_stages.geom_enable = true; + rctx->shader_stages.atom.dirty = true; + } + + /* gs_shader provides GS and VS (copy shader) */ + if (unlikely(rctx->geometry_shader.shader != rctx->gs_shader->current)) { + update_shader_atom(ctx, &rctx->geometry_shader, rctx->gs_shader->current); + update_shader_atom(ctx, &rctx->vertex_shader, rctx->gs_shader->current->gs_copy_shader); + /* Update clip misc state. */ + if (rctx->gs_shader->current->gs_copy_shader->pa_cl_vs_out_cntl != rctx->clip_misc_state.pa_cl_vs_out_cntl || + rctx->gs_shader->current->gs_copy_shader->shader.clip_dist_write != rctx->clip_misc_state.clip_dist_write) { + rctx->clip_misc_state.pa_cl_vs_out_cntl = rctx->gs_shader->current->gs_copy_shader->pa_cl_vs_out_cntl; + rctx->clip_misc_state.clip_dist_write = rctx->gs_shader->current->gs_copy_shader->shader.clip_dist_write; + rctx->clip_misc_state.atom.dirty = true; + } + } + + r600_shader_select(ctx, rctx->vs_shader, &vs_dirty); + if (unlikely(!rctx->vs_shader->current)) + return false; + + /* vs_shader is used as ES */ + if (unlikely(vs_dirty || rctx->export_shader.shader != rctx->vs_shader->current)) { + update_shader_atom(ctx, &rctx->export_shader, rctx->vs_shader->current); + } + } else { + if (unlikely(rctx->geometry_shader.shader)) { + update_shader_atom(ctx, &rctx->geometry_shader, NULL); + update_shader_atom(ctx, &rctx->export_shader, NULL); + rctx->shader_stages.geom_enable = false; + rctx->shader_stages.atom.dirty = true; + } - if (rctx->ps_shader && rctx->rasterizer && - ((rctx->rasterizer->sprite_coord_enable != rctx->ps_shader->current->sprite_coord_enable) || - (rctx->rasterizer->flatshade != rctx->ps_shader->current->flatshade))) { + r600_shader_select(ctx, rctx->vs_shader, &vs_dirty); + if (unlikely(!rctx->vs_shader->current)) + return false; - if (rctx->b.chip_class >= EVERGREEN) - evergreen_update_ps_state(ctx, rctx->ps_shader->current); - else - r600_update_ps_state(ctx, rctx->ps_shader->current); + if (unlikely(vs_dirty || rctx->vertex_shader.shader != rctx->vs_shader->current)) { + update_shader_atom(ctx, &rctx->vertex_shader, rctx->vs_shader->current); - ps_dirty = true; + /* Update clip misc state. */ + if (rctx->vs_shader->current->pa_cl_vs_out_cntl != rctx->clip_misc_state.pa_cl_vs_out_cntl || + rctx->vs_shader->current->shader.clip_dist_write != rctx->clip_misc_state.clip_dist_write) { + rctx->clip_misc_state.pa_cl_vs_out_cntl = rctx->vs_shader->current->pa_cl_vs_out_cntl; + rctx->clip_misc_state.clip_dist_write = rctx->vs_shader->current->shader.clip_dist_write; + rctx->clip_misc_state.atom.dirty = true; + } + } } - if (ps_dirty) { - rctx->pixel_shader.atom.num_dw = rctx->ps_shader->current->command_buffer.num_dw; - rctx->pixel_shader.atom.dirty = true; + r600_shader_select(ctx, rctx->ps_shader, &ps_dirty); + if (unlikely(!rctx->ps_shader->current)) + return false; + + if (unlikely(ps_dirty || rctx->pixel_shader.shader != rctx->ps_shader->current)) { + + if (rctx->cb_misc_state.nr_ps_color_outputs != rctx->ps_shader->current->nr_ps_color_outputs) { + rctx->cb_misc_state.nr_ps_color_outputs = rctx->ps_shader->current->nr_ps_color_outputs; + rctx->cb_misc_state.atom.dirty = true; + } + + if (rctx->b.chip_class <= R700) { + bool multiwrite = rctx->ps_shader->current->shader.fs_write_all; + + if (rctx->cb_misc_state.multiwrite != multiwrite) { + rctx->cb_misc_state.multiwrite = multiwrite; + rctx->cb_misc_state.atom.dirty = true; + } + } + + if (rctx->b.chip_class >= EVERGREEN) { + evergreen_update_db_shader_control(rctx); + } else { + r600_update_db_shader_control(rctx); + } + + if (unlikely(!ps_dirty && rctx->ps_shader && rctx->rasterizer && + ((rctx->rasterizer->sprite_coord_enable != rctx->ps_shader->current->sprite_coord_enable) || + (rctx->rasterizer->flatshade != rctx->ps_shader->current->flatshade)))) { + + if (rctx->b.chip_class >= EVERGREEN) + evergreen_update_ps_state(ctx, rctx->ps_shader->current); + else + r600_update_ps_state(ctx, rctx->ps_shader->current); + } + + update_shader_atom(ctx, &rctx->pixel_shader, rctx->ps_shader->current); } /* on R600 we stuff masks + txq info into one constant buffer */ @@ -1145,11 +1259,15 @@ static bool r600_update_derived_state(struct r600_context *rctx) r600_setup_buffer_constants(rctx, PIPE_SHADER_FRAGMENT); if (rctx->vs_shader && rctx->vs_shader->current->shader.uses_tex_buffers) r600_setup_buffer_constants(rctx, PIPE_SHADER_VERTEX); + if (rctx->gs_shader && rctx->gs_shader->current->shader.uses_tex_buffers) + r600_setup_buffer_constants(rctx, PIPE_SHADER_GEOMETRY); } else { if (rctx->ps_shader && rctx->ps_shader->current->shader.uses_tex_buffers) eg_setup_buffer_constants(rctx, PIPE_SHADER_FRAGMENT); if (rctx->vs_shader && rctx->vs_shader->current->shader.uses_tex_buffers) eg_setup_buffer_constants(rctx, PIPE_SHADER_VERTEX); + if (rctx->gs_shader && rctx->gs_shader->current->shader.uses_tex_buffers) + eg_setup_buffer_constants(rctx, PIPE_SHADER_GEOMETRY); } @@ -1157,6 +1275,8 @@ static bool r600_update_derived_state(struct r600_context *rctx) r600_setup_txq_cube_array_constants(rctx, PIPE_SHADER_FRAGMENT); if (rctx->vs_shader && rctx->vs_shader->current->shader.has_txq_cube_array_z_comp) r600_setup_txq_cube_array_constants(rctx, PIPE_SHADER_VERTEX); + if (rctx->gs_shader && rctx->gs_shader->current->shader.has_txq_cube_array_z_comp) + r600_setup_txq_cube_array_constants(rctx, PIPE_SHADER_GEOMETRY); if (rctx->b.chip_class < EVERGREEN && rctx->ps_shader && rctx->vs_shader) { if (!r600_adjust_gprs(rctx)) { @@ -1174,31 +1294,8 @@ static bool r600_update_derived_state(struct r600_context *rctx) rctx->blend_state.cso, blend_disable); } - return true; -} -static unsigned r600_conv_prim_to_gs_out(unsigned mode) -{ - static const int prim_conv[] = { - V_028A6C_OUTPRIM_TYPE_POINTLIST, - V_028A6C_OUTPRIM_TYPE_LINESTRIP, - V_028A6C_OUTPRIM_TYPE_LINESTRIP, - V_028A6C_OUTPRIM_TYPE_LINESTRIP, - V_028A6C_OUTPRIM_TYPE_TRISTRIP, - V_028A6C_OUTPRIM_TYPE_TRISTRIP, - V_028A6C_OUTPRIM_TYPE_TRISTRIP, - V_028A6C_OUTPRIM_TYPE_TRISTRIP, - V_028A6C_OUTPRIM_TYPE_TRISTRIP, - V_028A6C_OUTPRIM_TYPE_TRISTRIP, - V_028A6C_OUTPRIM_TYPE_LINESTRIP, - V_028A6C_OUTPRIM_TYPE_LINESTRIP, - V_028A6C_OUTPRIM_TYPE_TRISTRIP, - V_028A6C_OUTPRIM_TYPE_TRISTRIP, - V_028A6C_OUTPRIM_TYPE_TRISTRIP - }; - assert(mode < Elements(prim_conv)); - - return prim_conv[mode]; + return true; } void r600_emit_clip_misc_state(struct r600_context *rctx, struct r600_atom *atom) @@ -1227,7 +1324,7 @@ static void r600_draw_vbo(struct pipe_context *ctx, const struct pipe_draw_info return; } - if (!rctx->vs_shader) { + if (!rctx->vs_shader || !rctx->ps_shader) { assert(0); return; } @@ -1330,8 +1427,6 @@ static void r600_draw_vbo(struct pipe_context *ctx, const struct pipe_draw_info r600_write_context_reg(cs, R_028A0C_PA_SC_LINE_STIPPLE, S_028A0C_AUTO_RESET_CNTL(ls_mask) | (rctx->rasterizer ? rctx->rasterizer->pa_sc_line_stipple : 0)); - r600_write_context_reg(cs, R_028A6C_VGT_GS_OUT_PRIM_TYPE, - r600_conv_prim_to_gs_out(info.mode)); r600_write_config_reg(cs, R_008958_VGT_PRIMITIVE_TYPE, r600_conv_pipe_prim(info.mode)); @@ -1615,11 +1710,14 @@ bool sampler_state_needs_border_color(const struct pipe_sampler_state *state) void r600_emit_shader(struct r600_context *rctx, struct r600_atom *a) { + struct radeon_winsys_cs *cs = rctx->b.rings.gfx.cs; - struct r600_pipe_shader *shader = ((struct r600_shader_state*)a)->shader->current; + struct r600_pipe_shader *shader = ((struct r600_shader_state*)a)->shader; - r600_emit_command_buffer(cs, &shader->command_buffer); + if (!shader) + return; + r600_emit_command_buffer(cs, &shader->command_buffer); radeon_emit(cs, PKT3(PKT3_NOP, 0, 0)); radeon_emit(cs, r600_context_bo_reloc(&rctx->b, &rctx->b.rings.gfx, shader->bo, RADEON_USAGE_READ)); } @@ -1633,7 +1731,6 @@ struct pipe_surface *r600_create_surface_custom(struct pipe_context *pipe, assert(templ->u.tex.first_layer <= util_max_layer(texture, templ->u.tex.level)); assert(templ->u.tex.last_layer <= util_max_layer(texture, templ->u.tex.level)); - assert(templ->u.tex.first_layer == templ->u.tex.last_layer); if (surface == NULL) return NULL; pipe_reference_init(&surface->base.reference, 1); @@ -2148,6 +2245,7 @@ void r600_init_common_state_functions(struct r600_context *rctx) { rctx->b.b.create_fs_state = r600_create_ps_state; rctx->b.b.create_vs_state = r600_create_vs_state; + rctx->b.b.create_gs_state = r600_create_gs_state; rctx->b.b.create_vertex_elements_state = r600_create_vertex_fetch_shader; rctx->b.b.bind_blend_state = r600_bind_blend_state; rctx->b.b.bind_depth_stencil_alpha_state = r600_bind_dsa_state; @@ -2156,6 +2254,7 @@ void r600_init_common_state_functions(struct r600_context *rctx) rctx->b.b.bind_rasterizer_state = r600_bind_rs_state; rctx->b.b.bind_vertex_elements_state = r600_bind_vertex_elements; rctx->b.b.bind_vs_state = r600_bind_vs_state; + rctx->b.b.bind_gs_state = r600_bind_gs_state; rctx->b.b.delete_blend_state = r600_delete_blend_state; rctx->b.b.delete_depth_stencil_alpha_state = r600_delete_dsa_state; rctx->b.b.delete_fs_state = r600_delete_ps_state; @@ -2163,6 +2262,7 @@ void r600_init_common_state_functions(struct r600_context *rctx) rctx->b.b.delete_sampler_state = r600_delete_sampler_state; rctx->b.b.delete_vertex_elements_state = r600_delete_vertex_elements; rctx->b.b.delete_vs_state = r600_delete_vs_state; + rctx->b.b.delete_gs_state = r600_delete_gs_state; rctx->b.b.set_blend_color = r600_set_blend_color; rctx->b.b.set_clip_state = r600_set_clip_state; rctx->b.b.set_constant_buffer = r600_set_constant_buffer; diff --git a/src/gallium/drivers/r600/r600d.h b/src/gallium/drivers/r600/r600d.h index 89b09ddfa52..05d1f0ab34a 100644 --- a/src/gallium/drivers/r600/r600d.h +++ b/src/gallium/drivers/r600/r600d.h @@ -123,6 +123,7 @@ #define EVENT_TYPE_SO_VGTSTREAMOUT_FLUSH 0x1f #define EVENT_TYPE_SAMPLE_STREAMOUTSTATS 0x20 #define EVENT_TYPE_FLUSH_AND_INV_DB_META 0x2c /* supported on r700+ */ +#define EVENT_TYPE_VGT_FLUSH 0x24 #define EVENT_TYPE_FLUSH_AND_INV_CB_META 46 /* supported on r700+ */ #define EVENT_TYPE(x) ((x) << 0) #define EVENT_INDEX(x) ((x) << 8) @@ -200,6 +201,19 @@ /* Registers */ #define R_008490_CP_STRMOUT_CNTL 0x008490 #define S_008490_OFFSET_UPDATE_DONE(x) (((x) & 0x1) << 0) +#define R_008C40_SQ_ESGS_RING_BASE 0x008C40 +#define R_008C44_SQ_ESGS_RING_SIZE 0x008C44 +#define R_008C48_SQ_GSVS_RING_BASE 0x008C48 +#define R_008C4C_SQ_GSVS_RING_SIZE 0x008C4C +#define R_008C50_SQ_ESTMP_RING_BASE 0x008C50 +#define R_008C54_SQ_ESTMP_RING_SIZE 0x008C54 +#define R_008C50_SQ_GSTMP_RING_BASE 0x008C58 +#define R_008C54_SQ_GSTMP_RING_SIZE 0x008C5C + +#define R_0088C8_VGT_GS_PER_ES 0x0088C8 +#define R_0088CC_VGT_ES_PER_GS 0x0088CC +#define R_0088E8_VGT_GS_PER_VS 0x0088E8 + #define R_008960_VGT_STRMOUT_BUFFER_FILLED_SIZE_0 0x008960 /* read-only */ #define R_008964_VGT_STRMOUT_BUFFER_FILLED_SIZE_1 0x008964 /* read-only */ #define R_008968_VGT_STRMOUT_BUFFER_FILLED_SIZE_2 0x008968 /* read-only */ @@ -1824,12 +1838,20 @@ #define S_028A40_MODE(x) (((x) & 0x3) << 0) #define G_028A40_MODE(x) (((x) >> 0) & 0x3) #define C_028A40_MODE 0xFFFFFFFC +#define V_028A40_GS_OFF 0 +#define V_028A40_GS_SCENARIO_A 1 +#define V_028A40_GS_SCENARIO_B 2 +#define V_028A40_GS_SCENARIO_G 3 #define S_028A40_ES_PASSTHRU(x) (((x) & 0x1) << 2) #define G_028A40_ES_PASSTHRU(x) (((x) >> 2) & 0x1) #define C_028A40_ES_PASSTHRU 0xFFFFFFFB #define S_028A40_CUT_MODE(x) (((x) & 0x3) << 3) #define G_028A40_CUT_MODE(x) (((x) >> 3) & 0x3) #define C_028A40_CUT_MODE 0xFFFFFFE7 +#define V_028A40_GS_CUT_1024 0 +#define V_028A40_GS_CUT_512 1 +#define V_028A40_GS_CUT_256 2 +#define V_028A40_GS_CUT_128 3 #define R_008DFC_SQ_CF_WORD0 0x008DFC #define S_008DFC_ADDR(x) (((x) & 0xFFFFFFFF) << 0) #define G_008DFC_ADDR(x) (((x) >> 0) & 0xFFFFFFFF) @@ -2332,6 +2354,26 @@ #define S_028D44_ALPHA_TO_MASK_OFFSET3(x) (((x) & 0x3) << 14) #define S_028D44_OFFSET_ROUND(x) (((x) & 0x1) << 16) #define R_028868_SQ_PGM_RESOURCES_VS 0x028868 +#define R_028890_SQ_PGM_RESOURCES_ES 0x028890 +#define S_028890_NUM_GPRS(x) (((x) & 0xFF) << 0) +#define G_028890_NUM_GPRS(x) (((x) >> 0) & 0xFF) +#define C_028890_NUM_GPRS 0xFFFFFF00 +#define S_028890_STACK_SIZE(x) (((x) & 0xFF) << 8) +#define G_028890_STACK_SIZE(x) (((x) >> 8) & 0xFF) +#define C_028890_STACK_SIZE 0xFFFF00FF +#define S_028890_DX10_CLAMP(x) (((x) & 0x1) << 21) +#define G_028890_DX10_CLAMP(x) (((x) >> 21) & 0x1) +#define C_028890_DX10_CLAMP 0xFFDFFFFF +#define R_02887C_SQ_PGM_RESOURCES_GS 0x02887C +#define S_02887C_NUM_GPRS(x) (((x) & 0xFF) << 0) +#define G_02887C_NUM_GPRS(x) (((x) >> 0) & 0xFF) +#define C_02887C_NUM_GPRS 0xFFFFFF00 +#define S_02887C_STACK_SIZE(x) (((x) & 0xFF) << 8) +#define G_02887C_STACK_SIZE(x) (((x) >> 8) & 0xFF) +#define C_02887C_STACK_SIZE 0xFFFF00FF +#define S_02887C_DX10_CLAMP(x) (((x) & 0x1) << 21) +#define G_02887C_DX10_CLAMP(x) (((x) >> 21) & 0x1) +#define C_02887C_DX10_CLAMP 0xFFDFFFFF #define R_0286CC_SPI_PS_IN_CONTROL_0 0x0286CC #define R_0286D0_SPI_PS_IN_CONTROL_1 0x0286D0 #define R_028644_SPI_PS_INPUT_CNTL_0 0x028644 @@ -2421,11 +2463,15 @@ #define G_028C04_MAX_SAMPLE_DIST(x) (((x) >> 13) & 0xF) #define C_028C04_MAX_SAMPLE_DIST 0xFFFE1FFF #define R_0288CC_SQ_PGM_CF_OFFSET_PS 0x0288CC -#define R_0288DC_SQ_PGM_CF_OFFSET_FS 0x0288DC #define R_0288D0_SQ_PGM_CF_OFFSET_VS 0x0288D0 +#define R_0288D4_SQ_PGM_CF_OFFSET_GS 0x0288D4 +#define R_0288D8_SQ_PGM_CF_OFFSET_ES 0x0288D8 +#define R_0288DC_SQ_PGM_CF_OFFSET_FS 0x0288DC #define R_028840_SQ_PGM_START_PS 0x028840 #define R_028894_SQ_PGM_START_FS 0x028894 #define R_028858_SQ_PGM_START_VS 0x028858 +#define R_02886C_SQ_PGM_START_GS 0x02886C +#define R_028880_SQ_PGM_START_ES 0x028880 #define R_028080_CB_COLOR0_VIEW 0x028080 #define S_028080_SLICE_START(x) (((x) & 0x7FF) << 0) #define G_028080_SLICE_START(x) (((x) >> 0) & 0x7FF) @@ -2863,6 +2909,7 @@ #define R_0283F4_SQ_VTX_SEMANTIC_29 0x0283F4 #define R_0283F8_SQ_VTX_SEMANTIC_30 0x0283F8 #define R_0283FC_SQ_VTX_SEMANTIC_31 0x0283FC +#define R_0288C8_SQ_GS_VERT_ITEMSIZE 0x0288C8 #define R_0288E0_SQ_VTX_SEMANTIC_CLEAR 0x0288E0 #define R_028400_VGT_MAX_VTX_INDX 0x028400 #define S_028400_MAX_INDX(x) (((x) & 0xFFFFFFFF) << 0) @@ -3287,6 +3334,8 @@ #define R_028B28_VGT_STRMOUT_DRAW_OPAQUE_OFFSET 0x028B28 #define R_028B2C_VGT_STRMOUT_DRAW_OPAQUE_BUFFER_FILLED_SIZE 0x028B2C #define R_028B30_VGT_STRMOUT_DRAW_OPAQUE_VERTEX_STRIDE 0x028B30 +#define R_028B38_VGT_GS_MAX_VERT_OUT 0x028B38 /* r7xx */ +#define S_028B38_MAX_VERT_OUT(x) (((x) & 0x7FF) << 0) #define R_028B44_VGT_STRMOUT_BASE_OFFSET_HI_0 0x028B44 #define R_028B48_VGT_STRMOUT_BASE_OFFSET_HI_1 0x028B48 #define R_028B4C_VGT_STRMOUT_BASE_OFFSET_HI_2 0x028B4C diff --git a/src/gallium/drivers/r600/sb/sb_bc.h b/src/gallium/drivers/r600/sb/sb_bc.h index 73b8b08ba39..d03da98777d 100644 --- a/src/gallium/drivers/r600/sb/sb_bc.h +++ b/src/gallium/drivers/r600/sb/sb_bc.h @@ -169,8 +169,10 @@ enum shader_target { TARGET_UNKNOWN, TARGET_VS, + TARGET_ES, TARGET_PS, TARGET_GS, + TARGET_GS_COPY, TARGET_COMPUTE, TARGET_FETCH, diff --git a/src/gallium/drivers/r600/sb/sb_bc_dump.cpp b/src/gallium/drivers/r600/sb/sb_bc_dump.cpp index 9b1420d3895..1551e6d74f7 100644 --- a/src/gallium/drivers/r600/sb/sb_bc_dump.cpp +++ b/src/gallium/drivers/r600/sb/sb_bc_dump.cpp @@ -137,7 +137,7 @@ void bc_dump::dump(cf_node& n) { for (int k = 0; k < 4; ++k) s << chans[n.bc.sel[k]]; - } else if (n.bc.op_ptr->flags & (CF_STRM | CF_RAT)) { + } else if (n.bc.op_ptr->flags & CF_MEM) { static const char *exp_type[] = {"WRITE", "WRITE_IND", "WRITE_ACK", "WRITE_IND_ACK"}; fill_to(s, 18); @@ -150,6 +150,9 @@ void bc_dump::dump(cf_node& n) { if ((n.bc.op_ptr->flags & CF_RAT) && (n.bc.type & 1)) { s << ", @R" << n.bc.index_gpr << ".xyz"; } + if ((n.bc.op_ptr->flags & CF_MEM) && (n.bc.type & 1)) { + s << ", @R" << n.bc.index_gpr << ".x"; + } s << " ES:" << n.bc.elem_size; diff --git a/src/gallium/drivers/r600/sb/sb_bc_finalize.cpp b/src/gallium/drivers/r600/sb/sb_bc_finalize.cpp index 355eb63810c..54717f98a15 100644 --- a/src/gallium/drivers/r600/sb/sb_bc_finalize.cpp +++ b/src/gallium/drivers/r600/sb/sb_bc_finalize.cpp @@ -63,7 +63,7 @@ int bc_finalizer::run() { // workaround for some problems on r6xx/7xx // add ALU NOP to each vertex shader - if (!ctx.is_egcm() && sh.target == TARGET_VS) { + if (!ctx.is_egcm() && (sh.target == TARGET_VS || sh.target == TARGET_ES)) { cf_node *c = sh.create_clause(NST_ALU_CLAUSE); alu_group_node *g = sh.create_alu_group(); @@ -695,7 +695,7 @@ void bc_finalizer::finalize_cf(cf_node* c) { c->bc.rw_gpr = reg >= 0 ? reg : 0; c->bc.comp_mask = mask; - if ((flags & CF_RAT) && (c->bc.type & 1)) { + if (((flags & CF_RAT) || (!(flags & CF_STRM))) && (c->bc.type & 1)) { reg = -1; diff --git a/src/gallium/drivers/r600/sb/sb_bc_parser.cpp b/src/gallium/drivers/r600/sb/sb_bc_parser.cpp index 67e6c3a582d..7181e27acf0 100644 --- a/src/gallium/drivers/r600/sb/sb_bc_parser.cpp +++ b/src/gallium/drivers/r600/sb/sb_bc_parser.cpp @@ -58,7 +58,10 @@ int bc_parser::decode() { if (pshader) { switch (bc->type) { case TGSI_PROCESSOR_FRAGMENT: t = TARGET_PS; break; - case TGSI_PROCESSOR_VERTEX: t = TARGET_VS; break; + case TGSI_PROCESSOR_VERTEX: + t = pshader->vs_as_es ? TARGET_ES : TARGET_VS; + break; + case TGSI_PROCESSOR_GEOMETRY: t = TARGET_GS; break; case TGSI_PROCESSOR_COMPUTE: t = TARGET_COMPUTE; break; default: assert(!"unknown shader target"); return -1; break; } @@ -134,8 +137,12 @@ int bc_parser::parse_decls() { } } - if (sh->target == TARGET_VS) + if (sh->target == TARGET_VS || sh->target == TARGET_ES) sh->add_input(0, 1, 0x0F); + else if (sh->target == TARGET_GS) { + sh->add_input(0, 1, 0x0F); + sh->add_input(1, 1, 0x0F); + } bool ps_interp = ctx.hw_class >= HW_CLASS_EVERGREEN && sh->target == TARGET_PS; @@ -202,7 +209,7 @@ int bc_parser::decode_cf(unsigned &i, bool &eop) { if (cf->bc.rw_rel) gpr_reladdr = true; assert(!cf->bc.rw_rel); - } else if (flags & (CF_STRM | CF_RAT)) { + } else if (flags & CF_MEM) { if (cf->bc.rw_rel) gpr_reladdr = true; assert(!cf->bc.rw_rel); @@ -676,7 +683,7 @@ int bc_parser::prepare_ir() { } while (1); c->bc.end_of_program = eop; - } else if (flags & (CF_STRM | CF_RAT)) { + } else if (flags & CF_MEM) { unsigned burst_count = c->bc.burst_count; unsigned eop = c->bc.end_of_program; @@ -694,7 +701,7 @@ int bc_parser::prepare_ir() { sh->get_gpr_value(true, c->bc.rw_gpr, s, false); } - if ((flags & CF_RAT) && (c->bc.type & 1)) { // indexed write + if (((flags & CF_RAT) || (!(flags & CF_STRM))) && (c->bc.type & 1)) { // indexed write c->src.resize(8); for(int s = 0; s < 3; ++s) { c->src[4 + s] = diff --git a/src/gallium/drivers/r600/sb/sb_dump.cpp b/src/gallium/drivers/r600/sb/sb_dump.cpp index 7e317abe7a7..b2130a48fa6 100644 --- a/src/gallium/drivers/r600/sb/sb_dump.cpp +++ b/src/gallium/drivers/r600/sb/sb_dump.cpp @@ -349,7 +349,7 @@ void dump::dump_op(node &n, const char *name) { static const char *exp_type[] = {"PIXEL", "POS ", "PARAM"}; sblog << " " << exp_type[c->bc.type] << " " << c->bc.array_base; has_dst = false; - } else if (c->bc.op_ptr->flags & CF_STRM) { + } else if (c->bc.op_ptr->flags & (CF_MEM)) { static const char *exp_type[] = {"WRITE", "WRITE_IND", "WRITE_ACK", "WRITE_IND_ACK"}; sblog << " " << exp_type[c->bc.type] << " " << c->bc.array_base diff --git a/src/gallium/drivers/r600/sb/sb_shader.cpp b/src/gallium/drivers/r600/sb/sb_shader.cpp index 38617a85330..f996c0786d1 100644 --- a/src/gallium/drivers/r600/sb/sb_shader.cpp +++ b/src/gallium/drivers/r600/sb/sb_shader.cpp @@ -215,7 +215,7 @@ void shader::init() { void shader::init_call_fs(cf_node* cf) { unsigned gpr = 0; - assert(target == TARGET_VS); + assert(target == TARGET_VS || target == TARGET_ES); for(inputs_vec::const_iterator I = inputs.begin(), E = inputs.end(); I != E; ++I, ++gpr) { @@ -433,6 +433,7 @@ std::string shader::get_full_target_name() { const char* shader::get_shader_target_name() { switch (target) { case TARGET_VS: return "VS"; + case TARGET_ES: return "ES"; case TARGET_PS: return "PS"; case TARGET_GS: return "GS"; case TARGET_COMPUTE: return "COMPUTE"; |