出自Ogre3D开放资源地带

原文

目录 1 RT Shader System 1.1 Motivation 1.2 动机 1.3 Core features of the system 1.4 系统的核心特色 1.5 High level system overview 1.6 高级系统概览 1.7 Initializing the system 1.8 初始化系统 1.9 Creating shader based technique 1.10 创建shader based technique 1.11 Runtime shader generation 1.12 Creating custom shader s extensions 1.13 Finalizing the system 1.14 Known issues

RT Shader System

Motivation

动机

这些年，写[1]在3D项目开发中，已经日益成为一个非常普通的工作。大量的3D应用中的效果需要使用shader程序 Writing shading programs became a very common task when developing 3D based application during the last couple of years. Most of the visual effects used by 3D based applications involve shader programs. Here is just a short list of some common effects using shaders 这里仅仅是一些使用Shaders的特效的列表

• 硬件计算的动画
• 软阴影
• 法线/凹凸 贴图
• 环境贴图
• 高级的多重纹理特效
• Hardware animation
• Soft shadows
• Normal/Bump maps
• Specular maps
• Advanced multi-texturing effects

通常，手写shaders是个好的解决方式，一个基于目标场景自然特性的，可以完全掌控的shader代码很好优化。 Writing shaders by hand is in many cases the best solution – one has the full control of the shader code, optimizations based on the target scene nature can take place etc.

So why using the runtime shader system anyway ? 那么为什么还要使用RTSS呢？

• 节省开发时间 - 当你的目标场景是动态的（灯光类型和数量在改变）（雾在改变），材质特性的增长就会导致导致需要的shaders也显著增加。很快就会超过100，会成为一个很耗时的开发工作。
• Save development time – When your target scene is dynamic (light type and count changes, fog changes) and the number of material attributes increase the total count of needed shaders increase dramatically. It can easily cross the 100 count – and it becomes a time consuming development task.

可重用的代码 - 你可以根据不同的环境来扩展你的shader

• Reusable code - once you wrote shader extension you can use it anyware due to its independent nature.

自定义shaders扩展库 - 享受社区提供的开源特效库的快乐。跟手写shader代码不同，他们需要一些适配以便于融入你自己的shader代码，使用扩展库需要的改变最小。

• Custom shaders extension library - enjoy the shared library of effects created by the community. Unlike hand written shader code that require many adjustments in order to be plugged into your own shader code, using the extensions library require minimum changes.

Core features of the system

系统的核心特色

• Runtime shader generation synchronized with scene state. Each time scene settings changes a new set of shaders is generated.

• Full FFP (Fixed Function Pipeline) emulation. This feature is most useful combined with render system that doesn’t provide any FFP functionality (OpenGL ES 2.0, D3D10, D3D11 etc).

• Shader language independent interface – the logic representation of the shader programs is completely independent on the target shader language. You can generate different shader languages code from the same program.

• Pluggable interface allows extending the target shader languages set.

• Pluggable interface allows adding new shader based effect to the system in seamless way. Each effect code will be automatically combined with the rest of shader code.
• Smart program management – each shader program is created only once and may used by multiple passes.

• Automatic vertex shader compacting mechanism – no more compacting variables by hand.In case the amount of used vertex shader output registers exceeds the maximum allowed a compacting algorithm packs the vertex shader outputs and adds unpack code in the fragment shader side.

• Material script support both export and import.

根据场景状态实时动态生成shader。任何时候场景的设置改变，新的shader就会生成。

全FFP（固定管线）模拟。这个功能对于完全不再支持固定管线的渲染系统很有用（OpenGL ES 2.0, D3D10, D3D11等）

插件化的接口允许扩展目标shader语言集

插件化的接口允许无缝的添加新的shader。所有的特效代码都会自动的通过shader代码自动组合起来.

智能化的程序管理 - 每个shader程序只会创建一次，然后用于多次渲染绘制。

自动顶点shader压缩机制 - 不要需要人工压缩变量。这样，当使用的顶点shader输出寄存器的数量超过最大允许值时，就会使用一个压缩算法来打包顶点shader的输入，并在片段shader中增加解包的程序

支持材质脚本的导入和导出

High level system overview

高级系统概览

The main interface that will be used by users is the ShaderGenerator. 用户最主要用的接口是ShaderGenerator This singleton provides most of the services needed to generate shaders on the fly. 当需要生成shaders时，这个单间(singleton)提供了大量可使用的服务。 It let the user control the target shader code language (Currently Cg and GLSL supported), define the target profiles and of course generating shaders for a given material technique. 他允许用户决定目标shader语言（目前支持cg和GLSL），从一个给定的目标属性生成shadermaterial technique。

When the user asks the system to generate shaders for a given technique it has to provide the system a name for the target technique scheme. The system in turn, creates new technique based on the source technique but with different scheme name.

当用户需要系统生成shader的时候需要提供给系统一个名字来决定目标technique的样式(scheme)。系统运转方式是这样的：创建新的technique依赖于不同样式名称的源 technique 注意：为了避免冲突源technique不能包含任何shaders否则这步就会出错。 Note: In order to avoid clashes the source technique must NOT contain any shaders otherwise this step will fail.

The idea behind this concept is to use Ogre built in mechanism of material schemes – so all the user has to do in order to use the new technique is to change the material scheme of his viewport(s). 这个概念的动机是使用Ogre内建的材质样式机制 - 因此为了使用新的technique所有的用户必须要做的是改变他们自己视口的材质样式。 在每个视口更新系统描画之前，会进行所有相关联的shader的检查，这些shader是通过techniques来创建的。 Before each viewport update the system perform validation step of all associated shader based techniques it created. 这个不走包括自动同步场景的灯光和雾的状态。 This step includes automatic synchronization with the scene lights and fog states. 当系统确定样式已经过时的时候，他将会生成为每个technique生成最接近的shaser。 When the system detects that the scheme is out of date it generates the appropriate shaders for each technique.

The following steps are taken in order to generate shaders for a given technique. 以下步骤是通过一个给定的technique生成shader的过程

• For each pass in the technique the system builds a set of sub render states that describes the logic process of rendering pipeline from the draw call submission until the final pixel color.

1.对technique中的每个pass建立一个子渲染状态的集合，来描述渲染管线从绘制调用提交开始直到最后生成每个像素的颜色的逻辑过程。

• Each render state is translated to set of logic shader programs – currently only pixel and vertex shader.

The logic programs are then sent to specific shader language writer that produce source code in its shader language. The source code is used to create the GPU programs that are applied to the destination pass. Before rendering of an object that uses generated shaders the system allow each sub render state compose it to update the GPU constants associated with it. 2 每个渲染状态会被翻译成逻辑的shader程序的集合 - 目前只支持顶点着色和像素着色。 逻辑程序将被输送给特殊的shader语言程序，来生成源代码。这个源代码通常就是，创建最终能被目标通路接受的GPU程序。 In fact, the FFP emulation as well as the common shader based effects implemented using this methodology – meaning they override the SubRenderState and the SubRenderStateFactory. 事实上，FFP这个枚举，就是通常基础特效的shader用这种方法的实现 - 意味着他们改写(override)了SubRenderState 和 SubRenderStateFactor The FFP emulation contains the following SubRenderState overrides –

• FFPTransform – responsible for vertex transformation.
• FFPColour – responsible for colour registers.
• FFPLighting – responsible for lighting calculations.
• FFPTexturing – responsible for texture layers blending.
• FFPFog – responsible for fog effect.

• FFPTransform – 用于顶点变换
• FFPColour – 用于颜色的寄存器
• FFPLighting – 用于光照计算
• FFPTexturing – 用于材质混合
• FFPFog – 用于雾效果

Initializing the system

初始化系统

Initializing the system composed of the following steps – 步骤如下：

• 1.创建内部管理器和结构 通过调用Ogre::RTShader::initialize()
• 2.设置目标缓存路径 用来输出shaders或者读取之前应用程序已经生成过的shader
• 3.确认当前系统需要的shader库通过ResourceGroupManager::addResourceLocation方法添加到ResourceGroupManager了。
• 4.部署目标场景管理到shader生成器。
• Create the internal managers and structures via the Ogre::RTShader::initialize() method.
• Set the target cache path – this is the place on disk where the output shaders will be written to or will be read from in case they were generated by previous runs of your application.
• Verify that the location of the shader libs needed by the system added to the ResourceGroupManager via the ResourceGroupManager::addResourceLocation method.
• Assign the target scene manager to the shader generator.

			
if (Ogre::RTShader::ShaderGenerator::initialize())
{

// Grab the shader generator pointer.
mShaderGenerator = Ogre::RTShader::ShaderGenerator::getSingletonPtr();

// Add the shader libs resource location.
Ogre::ResourceGroupManager::getSingleton().addResourceLocation(shaderLibPath, "FileSystem");

// Set shader cache path.
mShaderGenerator->setShaderCachePath(shaderCachePath);		



// Set the scene manager.
mShaderGenerator->addSceneManager(sceneMgr);

			
				
return true;
}

Creating shader based technique

创建shader based technique

这步将把给定的technique和将要通过基本technique生成的shader联系起来。 调用ShaderGenerator::createShaderBasedTechnique，系统会生成与源technique相关的内部数据结构，还会增加新的technique到源材质中。 This step will associate the given technique with a destination shader generated based technique. Calling the ShaderGenerator::createShaderBasedTechnique will cause the system to generate internal data structures associated with the source technique and will add new technique to the source material. This new technique will have a scheme name that passed as an argument to this method and all its passes will contain shaders that the system will generate and update during the application runtime. Once you created such a technique all you have to do in order to use it is to change the material scheme of your viewport(s) to the same scheme name you passed as argument to this method.

// Create shader based technique from the default technique of the given material.
mShaderGenerator->createShaderBasedTechnique("Examples/BeachStones", Ogre::MaterialManager::DEFAULT_SCHEME_NAME, Ogre::RTShader::ShaderGenerator::DEFAULT_SCHEME_NAME);

// Apply the shader generated based techniques.
mViewport->setMaterialScheme(Ogre::RTShader::ShaderGenerator::DEFAULT_SCHEME_NAME);

Runtime shader generation

During the application runtime the ShaderGenerator instance receives notification on per frame basis from the it’s target SceneManager. At this point he checks the material scheme in use. In case the current scheme has representation at the manager he executes its validate method. The SGScheme validation includes synchronization with scene light and fog settings. In case it is out of date it will rebuild all shader generated techniques. The first step is to loop over every SGTechnique associated with this SGScheme and build its RenderStates - one for each pass. Each RenderState has its own hash code and it is cached at the ShaderGenerator. The same RenderState can be shared by multiple SGPasses. The second step is to loop again on every SGTechnique and acquire a program set for each SGPass. The actual acquiring process is done by the ProgramManager that generates CPU program representation, send them to a matching ProgramWriter that is chosen by the active target language, the writer generates source code that is the basis for the GPU programs. The result of this entire process is that each technique associated with the SGScheme has vertex and pixel shaders applied to all its passes. These shaders are synchronized with scene lights and fog settings.

File:RuntimeShaderGeneration.PNG

Creating custom shader s extensions

Although the system implements some common shader based effects such as per pixel lighting, normal map etc one may find it useful to write its own shader extensions.

In order to extend the system with your own shader effects you’ll have to follow these steps -

• Implement the SubRenderState interface – this is the main class that responsible for the actual effect processing such as preparing the destination pass, updating the CPU shader programs, updating the GPU shader parameters etc.

• Implement the SubRenderStateFactory interface – this class will allow the RTSS to create instances of the previous class via code or script as well as export it to material script file.

• Register the factory to the RTSS using the ShaderGenerator::addSubRenderStateFactory method.

• Add shader files that will supply all the actual shader functions your SubRenderState needs. In order to support multiple shader languages you should supply code for your entire desired target shading languages (CG, HLSL, GLSL etc). These files should be placed in a way that the resource manager could access them. This can be done by placing them in a valid resource location or by dynamically adding resource location.

Implementing the SubRenderState requires overriding the pure methods of the base class.

• SubRenderState::getType – should return unique string that identify the sub class implementation. That value is shared among all instances and can be stored in a static string variable. It uses to system to match between SubRenderState instance and the factory to should destroy it.
• SubRenderState::getExecutionOrder – should return integer value that will use the system to sort all SubRenderState instances of the same render state before each one of them will create its part in the CPU shader programs.
• SubRenderState::getHashCode – should return unsigned integer value that will represent the actual shader code it will generate. The first thing one should include in the hash code generation is the hash code of the type. If the SubRenderState is complex and different instances of it can generate different code sections it should reflect it in the returned hash code.

I.E – the FFPFog sub render state will return one hash code in case it configured to produce linear fog and second hash code if it configured to produce exponent based fog.

• SubRenderState::copyFrom – a simple copy method that uses the system when coping one instance to another.

Note: Only configuration data attributes should be copy here. SubRenderState::createCpuSubPrograms – this is the heart of this interface. This method should update the CPU shader programs with the specific details of the overriding class. The SubRenderState supply default implementation for this method which break down this method into three stages:

• resolving parameters – this stage should grab all the needed parameters for this SubRrenderState. In case of the FFPTransform it should resolve the world view projection matrix and vertex shader input and output position parameters.

Program* vsProgram = programSet->getCpuVertexProgram();
	
// Resolve World View Projection Matrix.
ParameterPtr wvpMatrix = vsProgram->resolveAutoParameterInt(GpuProgramParameters::ACT_WORLDVIEWPROJ_MATRIX, 0);
if (wvpMatrix.get() == NULL)
	return false;
		
Function* vsEntry = vsProgram->getEntryPointFunction();
assert(vsEntry != NULL);

// Resolve input position parameter.
ParameterPtr positionIn = vsEntry->resolveInputParameter(Parameter::SPS_POSITION, 0, Parameter::SPC_POSITION_OBJECT_SPACE, GCT_FLOAT4);	
if (positionIn.get() == NULL)
	return false;
	

// Resolve output position parameter.
ParameterPtr positionOut = vsEntry->resolveOutputParameter(Parameter::SPS_POSITION, 0, Parameter::SPC_POSITION_PROJECTIVE_SPACE, GCT_FLOAT4);
if (positionOut.get() == NULL)
   return false;

• resolving dependencies – this stage should provide the name of the external shader library files that contains the actual shader code needed by this SubRenderState.

In case of the FFPTexturing it will add the common and texturing library for both vertex and pixel shader program.

Program* vsProgram = programSet->getCpuVertexProgram();
Program* psProgram = programSet->getCpuFragmentProgram();

vsProgram->addDependency(FFP_LIB_COMMON);
vsProgram->addDependency(FFP_LIB_TEXTURING);	
psProgram->addDependency(FFP_LIB_COMMON);
psProgram->addDependency(FFP_LIB_TEXTURING);

• adding function invocations – this stage creates the function calls within this SubRenderState requires. Each function call has two keys that are used by the system to sort it before generating the actual shader code as well as set of in/out parameters.

A function invocation is added to either vertex shader program or fragment shader program. In case of the FFPFog it will add vertex depth calculation to the vertex shader program.

curFuncInvocation = OGRE_NEW FunctionInvocation(FFP_FUNC_PIXELFOG_DEPTH, FFP_VS_FOG, internalCounter++);
curFuncInvocation->pushOperand(mWorldViewProjMatrix, Operand::OPS_IN);
curFuncInvocation->pushOperand(mVSInPos, Operand::OPS_IN);	
curFuncInvocation->pushOperand(mVSOutDepth, Operand::OPS_OUT);	
vsMain->addAtomInstace(curFuncInvocation);		

Note:

• Each SubRenderState can add as many function invocations as it needs.
• Each SubRenderState can different function invocations in different ordering.
• The ordering of the function invocation is crucial – use the FFPVertexShaderStage and FFPFragmentShaderStage enumarations to place your invocations in the desired order.
• Make sure the parameter semantic (in/out) in the SubRenderState code matches to your shader code implementation you supplied in the library file. GLSL will fail to link to libray functions if it won’t be able to find a perfect function declaration match.

• SubRenderState::updateGpuProgramsParams – as the name suggest this method should be overridden only in case your SubRenderState should update some parameter it created before.

SubRenderState::preAddToRenderState – this method called before adding this SubRenderState to a parent RenderState instances. It allows this SubRenderState to exclude itself from the list in case the source pass is not matching. I.E – in case of SubRenderState that perform lighting calculations it can return false when the given source pass specifies that lighting calculations disabled for it.

if (srcPass->getLightingEnabled() == false)
	return false;

This method also let the SubRenderState to opportunity to modify the destination pass. I.E the NormalMapLighting instance adds the normal map texture unit in this context.

Implementing the SunRenderStateFactory is much simpler and involves implementing the following methods

• SubRenderStateFactory:: createInstanceImpl – this method should return instance for the SubRenderState sub class.
• SubRenderStateFactory:: createInstance – this method should return instasnce for the SubRenderState sub class using the given script compiler parameters. Implemet this method if you want to be able to creat your custom shader extension from material script.
• SubRenderStateFactory ::writeInstance – this method should write down the parameters of a given SubRenderState instance to material script file. Implement this method if you want to be able to export a material that contains your custom shader extension.

Finalizing the system

Coming soon...

Known issues

Coming soon...