d3d1x: add new Direct3D 10/11 COM state tracker for Gallium

This is a new implementation of the Direct3D 11 COM API for Gallium.

Direct3D 10 and 10.1 implementations are also provided, which are
automatically generated with s/D3D11/D3D10/g plus a bunch of #ifs.

While this is an initial version, most of the code is there (limited
to what Gallium can express), and tri, gears and texturing demos
are working.

The primary goal is to realize Gallium's promise of multiple API
support, and provide an API that can be easily implemented with just
a very thin wrapper over Gallium, instead of the enormous amount of
complex code needed for OpenGL.

The secondary goal is to run Windows Direct3D 10/11 games on Linux
using Wine.
Wine dlls are currently not provided, but adding them should be
quite easy.

Fglrx and nvidia drivers can also be supported by writing a Gallium
driver that talks to them using OpenGL, which is a relatively easy
task.
Thanks to the great design of Direct3D 10/11 and closeness to Gallium,
this approach should not result in detectable overhead, and is the
most maintainable way to do it, providing a path to switch to the
open Gallium drivers once they are on par with the proprietary ones.

Currently Wine has a very limited Direct3D 10 implementation, and
completely lacks a Direct3D 11 implementation.

Note that Direct3D 10/11 are completely different from Direct3D 9
and earlier, and thus warrant a fully separate implementation.

The third goal is to provide a superior alternative to OpenGL for
graphics programming on non-Windows systems, particularly Linux
and other free and open systems.

Thanks to a very clean and well-though design done from scratch,
the Direct3D 10/11 APIs are vastly better than OpenGL and can be
supported with orders of magnitude less code and development time,
as you can see by comparing the lines of code of this commit and
those in the existing Mesa OpenGL implementation.

This would have been true for the Longs Peak proposal as well, but
unfortunately it was abandoned by Khronos, leaving the OpenGL
ecosystem without a graphics API with a modern design.

A binding of Direct3D 10/11 to EGL would solve this issue in the
most economical way possible, and this would be great to provide
in Mesa, since DXGI, the API used to bind Direct3D 10/11 to Windows,
is a bit suboptimal, especially on non-Windows platforms.

Finally, a mature Direct3D 10/11 implementation is intrinsically going
to be faster and more reliable than an OpenGL implementation, thanks
to the dramatically smaller API and the segregation of all nontrivial
work to object creation that the application must perform ahead of
time.

Currently, this commit contains:
- Independently created headers for Direct3D 10, 10.1, 11 and DXGI 1.1,
  partially based on the existing Wine headers for D3D10 and DXGI 1.0
- A parser for Direct3D 10/11 DXBC and TokenizedProgramFormat (TPF)
- A shader translator from TokenizedProgramFormat to TGSI
- Implementation of the Direct3D 11 core interfaces
- Automatically generated implementation of Direct3D 10 and 10.1
- Implementation of DXGI using the "native" framework of the EGL st
- Demos, usable either on Windows or on this implementation
  - d3d11tri, a clone of tri
  - d3d11tex, a (multi)texturing demo
  - d3d11gears, an improved version of glxgears
  - d3d11spikysphere, a D3D11 tessellation demo (currently Windows-only)
- A downloader for the Microsoft HLSL compiler, needed to recompile
  the shaders (compiled shader bytecode is also included)

To compile this, configure at least with these options:
--with-state-trackers=egl,d3d1x --with-egl-platforms=x11
plus some gallium drivers (such as softpipe with --enable-gallium-swrast)

The Wine headers (usually from a wine-dev or wine-devel package) must
be installed.
Only x86-32 has been tested.

You may need to run "make" in the subdirectories of src/gallium/winsys/sw
and you may need to manually run "sudo make install" in
src/gallium/targets/egl

To test it, run the demos in the "progs" directory.
Windows binaries are included to find out how demos should work, and to
test Wine integration when it will be done.

Enjoy, and let me know if you manage to compile and run this, or
which issues you are facing if not.

Using softpipe is recommended for now, and your mileage with hardware
drivers may vary.
However, getting this to work on hardware drivers is also obviously very
important.

Note that currently llvmpipe is buggy and causes all 3 gears to be
drawn with the same color.
Use export GALLIUM_DRIVER=softpipe to avoid this.

Thanks to all the Gallium contributors and especially the VMware
team, whose work made it possible to implement Direct3D 10/11 much
more easily than it would have been otherwise.

1 files changed, 424 insertions, 0 deletions

diff --git a/src/gallium/state_trackers/d3d1x/d3d1xshader/src/tpf_parse.cpp b/src/gallium/state_trackers/d3d1x/d3d1xshader/src/tpf_parse.cpp
new file mode 100644
index 00000000000..a4f6cc6b661
--- /dev/null
+++ b/src/gallium/state_trackers/d3d1x/d3d1xshader/src/tpf_parse.cpp
@@ -0,0 +1,424 @@
+/**************************************************************************
+ *
+ * Copyright 2010 Luca Barbieri
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining
+ * a copy of this software and associated documentation files (the
+ * "Software"), to deal in the Software without restriction, including
+ * without limitation the rights to use, copy, modify, merge, publish,
+ * distribute, sublicense, and/or sell copies of the Software, and to
+ * permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the
+ * next paragraph) shall be included in all copies or substantial
+ * portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+ * IN NO EVENT SHALL THE COPYRIGHT OWNER(S) AND/OR ITS SUPPLIERS BE
+ * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+ * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+ * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+ *
+ **************************************************************************/
+
+#include "tpf.h"
+#include "utils.h"
+
+#if 1
+#define check(x) assert(x)
+#define fail(x) assert(0 && (x))
+#else
+#define check(x) do {if(!(x)) throw(#x);} while(0)
+#define fail(x) throw(x)
+#endif
+
+struct tpf_parser
+{
+	unsigned* tokens;
+	unsigned* tokens_end;
+	tpf_program& program;
+
+	tpf_parser(tpf_program& program, void* p_tokens, unsigned size)
+	: program(program)
+	{
+		tokens = (unsigned*)p_tokens;
+		tokens_end = (unsigned*)((char*)p_tokens + size);
+	}
+
+	/* TODO: byteswap if machine is big endian */
+	uint32_t read32()
+	{
+		check(tokens < tokens_end);
+		return bswap_le32(*tokens++);
+	}
+
+	template<typename T>
+	void read_token(T* tok)
+	{
+		*(unsigned*)tok = read32();
+	}
+
+	uint64_t read64()
+	{
+		unsigned a = read32();
+		unsigned b = read32();
+		return (uint64_t)a | ((uint64_t)b << 32);
+	}
+
+	void skip(unsigned toskip)
+	{
+		tokens += toskip;
+	}
+
+	void read_op(tpf_op* pop)
+	{
+		tpf_op& op = *pop;
+		tpf_token_operand optok;
+		read_token(&optok);
+		assert(optok.file < TPF_FILE_COUNT);
+		op.swizzle[0] = 0;
+		op.swizzle[1] = 1;
+		op.swizzle[2] = 2;
+		op.swizzle[3] = 3;
+		op.mask = 0xf;
+		switch(optok.comps_enum)
+		{
+		case TPF_OPERAND_COMPS_0:
+			op.comps = 0;
+			break;
+		case TPF_OPERAND_COMPS_1:
+			op.comps = 1;
+			break;
+		case TPF_OPERAND_COMPS_4:
+			op.comps = 4;
+			op.mode = optok.mode;
+			switch(optok.mode)
+			{
+			case TPF_OPERAND_MODE_MASK:
+				op.mask = TPF_OPERAND_SEL_MASK(optok.sel);
+				break;
+			case TPF_OPERAND_MODE_SWIZZLE:
+				op.swizzle[0] = TPF_OPERAND_SEL_SWZ(optok.sel, 0);
+				op.swizzle[1] = TPF_OPERAND_SEL_SWZ(optok.sel, 1);
+				op.swizzle[2] = TPF_OPERAND_SEL_SWZ(optok.sel, 2);
+				op.swizzle[3] = TPF_OPERAND_SEL_SWZ(optok.sel, 3);
+				break;
+			case TPF_OPERAND_MODE_SCALAR:
+				op.swizzle[0] = op.swizzle[1] = op.swizzle[2] = op.swizzle[3] = TPF_OPERAND_SEL_SCALAR(optok.sel);
+				break;
+			}
+			break;
+		case TPF_OPERAND_COMPS_N:
+			fail("Unhandled operand component type");
+		}
+		op.file = (tpf_file)optok.file;
+		op.num_indices = optok.num_indices;
+
+		if(optok.extended)
+		{
+			tpf_token_operand_extended optokext;
+			read_token(&optokext);
+			if(optokext.type == 0)
+			{}
+			else if(optokext.type == 1)
+			{
+				op.neg = optokext.neg;
+				op.abs= optokext.abs;
+			}
+			else
+				fail("Unhandled extended operand token type");
+		}
+
+		for(unsigned i = 0; i < op.num_indices; ++i)
+		{
+			unsigned repr;
+			if(i == 0)
+				repr = optok.index0_repr;
+			else if(i == 1)
+				repr = optok.index1_repr;
+			else if(i == 2)
+				repr = optok.index2_repr;
+			else
+				fail("Unhandled operand index representation");
+			op.indices[0].disp = 0;
+			// TODO: is disp supposed to be signed here??
+			switch(repr)
+			{
+			case TPF_OPERAND_INDEX_REPR_IMM32:
+				op.indices[i].disp = (int32_t)read32();
+				break;
+			case TPF_OPERAND_INDEX_REPR_IMM64:
+				op.indices[i].disp = read64();
+				break;
+			case TPF_OPERAND_INDEX_REPR_REG:
+relative:
+				op.indices[i].reg.reset(new tpf_op());
+				read_op(&*op.indices[0].reg);
+				break;
+			case TPF_OPERAND_INDEX_REPR_REG_IMM32:
+				op.indices[i].disp = (int32_t)read32();
+				goto relative;
+			case TPF_OPERAND_INDEX_REPR_REG_IMM64:
+				op.indices[i].disp = read64();
+				goto relative;
+			}
+		}
+
+		if(op.file == TPF_FILE_IMMEDIATE32)
+		{
+			for(unsigned i = 0; i < op.comps; ++i)
+				op.imm_values[i].i32 = read32();
+		}
+		else if(op.file == TPF_FILE_IMMEDIATE64)
+		{
+			for(unsigned i = 0; i < op.comps; ++i)
+				op.imm_values[i].i64 = read64();
+		}
+	}
+
+	void do_parse()
+	{
+		read_token(&program.version);
+
+		unsigned lentok = read32();
+		tokens_end = tokens - 2 + lentok;
+
+		while(tokens != tokens_end)
+		{
+			tpf_token_instruction insntok;
+			read_token(&insntok);
+			unsigned* insn_end = tokens - 1 + insntok.length;
+			tpf_opcode opcode = (tpf_opcode)insntok.opcode;
+			check(opcode < TPF_OPCODE_D3D11_COUNT);
+
+			if(opcode == TPF_OPCODE_CUSTOMDATA)
+			{
+				unsigned customlen = read32() - 2;
+				skip(customlen);
+				continue;
+			}
+
+			if((opcode >= TPF_OPCODE_DCL_RESOURCE && opcode <= TPF_OPCODE_DCL_GLOBAL_FLAGS)
+				|| (opcode >= TPF_OPCODE_DCL_STREAM && opcode <= TPF_OPCODE_DCL_RESOURCE_STRUCTURED))
+			{
+				tpf_dcl& dcl = *new tpf_dcl;
+				program.dcls.push_back(&dcl);
+				(tpf_token_instruction&)dcl = insntok;
+
+				tpf_token_instruction_extended exttok;
+				memcpy(&exttok, &insntok, sizeof(exttok));
+				while(exttok.extended)
+				{
+					read_token(&exttok);
+				}
+
+#define READ_OP_ANY dcl.op.reset(new tpf_op()); read_op(&*dcl.op);
+#define READ_OP(FILE) READ_OP_ANY
+				//check(dcl.op->file == TPF_FILE_##FILE);
+
+				switch(opcode)
+				{
+				case TPF_OPCODE_DCL_GLOBAL_FLAGS:
+					break;
+				case TPF_OPCODE_DCL_RESOURCE:
+					READ_OP(RESOURCE);
+					read_token(&dcl.rrt);
+					break;
+				case TPF_OPCODE_DCL_SAMPLER:
+					READ_OP(SAMPLER);
+					break;
+				case TPF_OPCODE_DCL_INPUT:
+				case TPF_OPCODE_DCL_INPUT_PS:
+					READ_OP(INPUT);
+					break;
+				case TPF_OPCODE_DCL_INPUT_SIV:
+				case TPF_OPCODE_DCL_INPUT_SGV:
+				case TPF_OPCODE_DCL_INPUT_PS_SIV:
+				case TPF_OPCODE_DCL_INPUT_PS_SGV:
+					READ_OP(INPUT);
+					dcl.sv = (tpf_sv)(uint16_t)read32();
+					break;
+				case TPF_OPCODE_DCL_OUTPUT:
+					READ_OP(OUTPUT);
+					break;
+				case TPF_OPCODE_DCL_OUTPUT_SIV:
+				case TPF_OPCODE_DCL_OUTPUT_SGV:
+					READ_OP(OUTPUT);
+					dcl.sv = (tpf_sv)(uint16_t)read32();
+					break;
+				case TPF_OPCODE_DCL_INDEX_RANGE:
+					READ_OP_ANY;
+					check(dcl.op->file == TPF_FILE_INPUT || dcl.op->file == TPF_FILE_OUTPUT);
+					dcl.num = read32();
+					break;
+				case TPF_OPCODE_DCL_TEMPS:
+					dcl.num = read32();
+					break;
+				case TPF_OPCODE_DCL_INDEXABLE_TEMP:
+					READ_OP(INDEXABLE_TEMP);
+					dcl.indexable_temp.num = read32();
+					dcl.indexable_temp.comps = read32();
+					break;
+				case TPF_OPCODE_DCL_CONSTANT_BUFFER:
+					READ_OP(CONSTANT_BUFFER);
+					break;
+				case TPF_OPCODE_DCL_GS_INPUT_PRIMITIVE:
+				case TPF_OPCODE_DCL_GS_OUTPUT_PRIMITIVE_TOPOLOGY:
+					break;
+				case TPF_OPCODE_DCL_MAX_OUTPUT_VERTEX_COUNT:
+					dcl.num = read32();
+					break;
+				case TPF_OPCODE_DCL_GS_INSTANCE_COUNT:
+					dcl.num = read32();
+					break;
+				case TPF_OPCODE_DCL_INPUT_CONTROL_POINT_COUNT:
+				case TPF_OPCODE_DCL_OUTPUT_CONTROL_POINT_COUNT:
+				case TPF_OPCODE_DCL_TESS_DOMAIN:
+				case TPF_OPCODE_DCL_TESS_PARTITIONING:
+				case TPF_OPCODE_DCL_TESS_OUTPUT_PRIMITIVE:
+					break;
+				case TPF_OPCODE_DCL_HS_MAX_TESSFACTOR:
+					dcl.f32 = read32();
+					break;
+				case TPF_OPCODE_DCL_HS_FORK_PHASE_INSTANCE_COUNT:
+					dcl.num = read32();
+					break;
+				case TPF_OPCODE_DCL_FUNCTION_BODY:
+					dcl.num = read32();
+					break;
+				case TPF_OPCODE_DCL_FUNCTION_TABLE:
+					dcl.num = read32();
+					dcl.data = malloc(dcl.num * sizeof(uint32_t));
+					for(unsigned i = 0; i < dcl.num; ++i)
+						((uint32_t*)dcl.data)[i] = read32();
+					break;
+				case TPF_OPCODE_DCL_INTERFACE:
+					dcl.intf.id = read32();
+					dcl.intf.expected_function_table_length = read32();
+					{
+						uint32_t v = read32();
+						dcl.intf.table_length = v & 0xffff;
+						dcl.intf.array_length = v >> 16;
+					}
+					dcl.data = malloc(dcl.intf.table_length * sizeof(uint32_t));
+					for(unsigned i = 0; i < dcl.intf.table_length; ++i)
+						((uint32_t*)dcl.data)[i] = read32();
+					break;
+				case TPF_OPCODE_DCL_THREAD_GROUP:
+					dcl.thread_group_size[0] = read32();
+					dcl.thread_group_size[1] = read32();
+					dcl.thread_group_size[2] = read32();
+					break;
+				case TPF_OPCODE_DCL_UNORDERED_ACCESS_VIEW_TYPED:
+					READ_OP(UNORDERED_ACCESS_VIEW);
+					read_token(&dcl.rrt);
+					break;
+				case TPF_OPCODE_DCL_UNORDERED_ACCESS_VIEW_RAW:
+					READ_OP(UNORDERED_ACCESS_VIEW);
+					break;
+				case TPF_OPCODE_DCL_UNORDERED_ACCESS_VIEW_STRUCTURED:
+					READ_OP(UNORDERED_ACCESS_VIEW);
+					dcl.structured.stride = read32();
+					break;
+				case TPF_OPCODE_DCL_THREAD_GROUP_SHARED_MEMORY_RAW:
+					READ_OP(THREAD_GROUP_SHARED_MEMORY);
+					dcl.num = read32();
+					break;
+				case TPF_OPCODE_DCL_THREAD_GROUP_SHARED_MEMORY_STRUCTURED:
+					READ_OP(THREAD_GROUP_SHARED_MEMORY);
+					dcl.structured.stride = read32();
+					dcl.structured.count = read32();
+					break;
+				case TPF_OPCODE_DCL_RESOURCE_RAW:
+					READ_OP(RESOURCE);
+					break;
+				case TPF_OPCODE_DCL_RESOURCE_STRUCTURED:
+					READ_OP(RESOURCE);
+					dcl.structured.stride = read32();
+					break;
+				case TPF_OPCODE_DCL_STREAM:
+					/* TODO: dcl_stream is undocumented: what is it? */
+					fail("Unhandled dcl_stream since it's undocumented");
+				default:
+					fail("Unhandled declaration");
+				}
+
+				check(tokens == insn_end);
+			}
+			else
+			{
+				tpf_insn& insn = *new tpf_insn;
+				program.insns.push_back(&insn);
+				(tpf_token_instruction&)insn = insntok;
+
+				tpf_token_instruction_extended exttok;
+				memcpy(&exttok, &insntok, sizeof(exttok));
+				while(exttok.extended)
+				{
+					read_token(&exttok);
+					if(exttok.type == TPF_TOKEN_INSTRUCTION_EXTENDED_TYPE_SAMPLE_CONTROLS)
+					{
+						insn.sample_offset[0] = exttok.sample_controls.offset_u;
+						insn.sample_offset[1] = exttok.sample_controls.offset_v;
+						insn.sample_offset[2] = exttok.sample_controls.offset_w;
+					}
+					else if(exttok.type == TPF_TOKEN_INSTRUCTION_EXTENDED_TYPE_RESOURCE_DIM)
+						insn.resource_target = exttok.resource_target.target;
+					else if(exttok.type == TPF_TOKEN_INSTRUCTION_EXTENDED_TYPE_RESOURCE_RETURN_TYPE)
+					{
+						insn.resource_return_type[0] = exttok.resource_return_type.x;
+						insn.resource_return_type[1] = exttok.resource_return_type.y;
+						insn.resource_return_type[2] = exttok.resource_return_type.z;
+						insn.resource_return_type[3] = exttok.resource_return_type.w;
+					}
+				}
+
+				switch(opcode)
+				{
+				case TPF_OPCODE_INTERFACE_CALL:
+					insn.num = read32();
+					break;
+				default:
+					break;
+				}
+
+				unsigned op_num = 0;
+				while(tokens != insn_end)
+				{
+					check(tokens < insn_end);
+					check(op_num < TPF_MAX_OPS);
+					insn.ops[op_num].reset(new tpf_op);
+					read_op(&*insn.ops[op_num]);
+					++op_num;
+				}
+				insn.num_ops = op_num;
+			}
+		}
+	}
+
+	const char* parse()
+	{
+		try
+		{
+			do_parse();
+			return 0;
+		}
+		catch(const char* error)
+		{
+			return error;
+		}
+	}
+};
+
+tpf_program* tpf_parse(void* tokens, int size)
+{
+	tpf_program* program = new tpf_program;
+	tpf_parser parser(*program, tokens, size);
+	if(!parser.parse())
+		return program;
+	delete program;
+	return 0;
+}