Light PrePass: Neon Lights !

A couple of days ago I was wondering what kind of new light types I would be able to model with the Light Prepass renderer that I wouldn't be able to implement easily with a standard forward renderer...

I came up with the simple idea of neon lights I could model with a line segment and a radius of influence. The main algorithm would be to find for each fragment the closest point on the neon segment which is easily done in the pixel shader :

float4 findClosestPointOnSegemntFromPoint(
    float4 vA,        // start of the neon segment
    float4 vB,        // end of the neon segment
    float vP          // point to light    )
{
    float4 dir = normalize( vB - vA );      // direction vector of the segment
    float len = distance( vA, vB );         // length of the segment

    float t = dot( (P - vA), dir );         // distance of the closest point P on the seg from the origin
    float normalized_t = t / len;           // points belonging to the segment will be in [0;1]
    normalized_t = saturate( normalized_t );// discatding points not belonging to the segment

    float4 vClosestPoint = vA + dir * len * normalized_t;        // Closest point !
   
    return vClosestPoint;
}

 Then I simply use this point as if it was the source of a point light. The results are actually quite nice:

The main issue will be to cast shadows from this type of lights but I can probably live without.

In conclusion, deferred rendering is definitely cool!

Light PrePass: Spot Lights !

This one will be a brief post, just wanted to share the first 2 spotlights of the renderer :)
Here they are:

Just need to add a nice angular falloff and this is good to go ... (and I should maybe make a nicer test scene because this one is really ugly)

-m

Light PrePass Renderer update

I resumed working on the light prepass renderer, trying to solve a few issues before adding more features to it.

The first thing was to use a sphere mesh for point lights instead of a fullscreen pass, in order to dramatically cut down the fillrate requirements. A fullscreen pass would still be faster if the camera is inside the light but on a general case, I assume that this case will be rare and didn't implement a special case for it (I might add that as an optim if needed later on).

For that to work I had to change the shaders a bit because the view space position reconstruction code only worked for fullscreen quads. I currently use the VPOS HLSL semantic to get the screen position of the current pixel and from there compute proper UV and ray vector. This works well but I would prefer not to rely on VPOS so I'll fix the shaders later on to generate the screen positions myself in the vertex shader.


Next thing I noticed is that it makes things simpler to only draw the backface triangles of my sphere as this works even if the camera is inside the light (all that is kinda similar to the old shadow volumes issues, Carmack's reverse, etc ...). Another issue may rise when the sphere gets clipped by the far plane. To fix that, I added the following line to the vertex shader that transforms his geometry:

    VS_OUTPUT main_LPP_PointLight( VS_INPUT _Input )
    {

        float4 vScreenPosition = mul( float4(_Input.vPosition, 1), g_mWorldViewProj );
        vScreenPosition.w = vScreenPosition.z;

        output.vPos = vScreenPosition;

        return output;
    }

This projects any vertex that is behind the far plane on the farplane and caps the hole that would have resulted in far clipping parts of the mesh. This is achievd by making sure that z/w = 1 all the time. Again, this used to be a shadow volume trick I think ...

Then, I found another artefact that I had to fix, cf the image below:

This time the culprit was the depth target generated during the geometry pass. Basically, the buffer was cleared to 0, which would map all the pixels to the far plane value ... creating a sort of solid wall at the end of the view frustum. And if a light got close to the far plane, it would start to light these pixels. The fix for that was to be able to discard pixels that were not written during the geometry pass. Ideally clearing the target with a value of -1 would be ideal but this is not supported by the Clear() method available in DX9 as the value gets clamped in the range [0;1], To simulate that a simple trick is:

• Clear the depth buffer to 0

     Device::GetInstance().Clear( ClearFlag_All, 0x00000000, 1.0f, 0 );

• store the depth in [1;2] instead of [0;1] in the geometry pass output

     PS_OUTPUT main_LPP_GeometryPass( VS_OUTPUT _Input )

     {

          ...

          output.RT0_Color = -_Input.vPosition.z / g_vProjectionAttributes.w + 1;     

          ...

     }

• subtract 1 to the value we read back from the depth target. if this pixel was written, we get a nice depth in [0;1]. If not, we get -1 and we can clip this pixel 

     float4 main_LPP_PointLight( PS_INPUT _Input ) : COLOR0
     {

          ...
          float    depth = tex2D( g_tDepthBuffer, uv ).x;              
         depth = depth - 1;
         clip( depth );

          ...
     }

With these fixes, I thing I'm good with the basics for the point lights and most of it should be applicable to spot lights as well ... Back to work !

Texel to Pixel Centers ... and a Splash screen !

After a pretty intense period at work, things are nicely settling down and I found some time to work on my various things on the engine during the last couple of days.

I mainly worked on some ground work inside the resource management code in order to make sure I can create any type of resource using the serialization/reflection code but also by simply providing a block of data in memory, independently of where the data is coming from. This sounds pretty simple and obvious but enforcing that for all types of binary resources gives you a nice and flexible code architecture and makes things more separated and testable.

About the renderer, I noticed some rendering artifacts on the light pass and started investigating where this was coming from. I first suspected issues with my shader code as I am no expert but it turns out that the culprit was a well known fact about D3D9: Texel to Pixel centers misalignement. Instead of me attempting to explain the issue and solution, here is the official page from D3D9's online help about that matter:
http://msdn.microsoft.com/en-us/library/bb219690%28VS.85%29.aspx

And here are 2 shots, the left one showing the artifacts and the right one showing the result once the bug fixed:

Finally, I spent some time looking at how threads are implemented in C#, what is the interface exposed to the user etc ... As a simple way to use that knowledge, I implemented a splash screen for the editor, that is updated in a separate thread. Nothing particularly hard here but I am really pleased with the result ... probably the best my somewhat limited artistic talent can produce:

That's all for today, hoping to find more time to spend on adding more stuff to the renderer and then move on.

Light PrePass in the editor

I made some fixes so I can have more than 1 light and their contribution nicely accumulates in the light buffer. Moreover, I fixed some minor bugs so I can use all that in my editor and move the camera and lights easily. It's not that much but I'll just put a screenshot :

I hope I'll have some more time this week to work on point and directional lights as well as improving the point lights (by drawing a 3D sphere or a 2D billboard instead of a fullscreen quad). And also more time to posts details about all that, with code snippets maybe :)

-m

Lessons learnt while writing shaders

I wrote more shader code over the past week than I ever did before that, and there are several things I learnt while doing so about that. I thought that might be useful to share that even if some are straightforward or just common sense.

First of all, PIX is your friend. I repeat: PIX is your friend! Make sure you can run this tool with your app and get the vertex and pixel shader debugging features working, this is invaluable... That being said, PIX is a little bit whimsical at times so you need to make sure that your app is complian with PIX, ie:

• Shaders are compiled in debug mode so you don't have to debug the assembly output but your actual code. This is done by passing D3DXSHADER_DEBUG | D3DXSHADER_SKIPOPTIMIZATION as flags to D3DXCompileShader(...). 
• Having 0 warnings while compiling your shaders improves stability
• If you experience troubles, you can try switching to softvare vertex processing or using the reference device (whis is going to be super slow but helps catching some bugs)

Next thing is about 3D maths and how it is done on computers - both CPU and GPU... There are many reasons why maths can go wrong down the chain if you're not careful. The kind of questions I tend to ask myself to try and prove me wrong and then eventually right when I write 3D maths includes:

• Which coordinate system this Matrix/Vector is into?
• Should I do Matrix * Vector or Vector * Matrix ??
• Should I use this Matrix or it's inverse? Transpose?
• Is it Z-up or Y-up?
• Is it a left handed or right handed coordinate system?
• In that float4, what should the W component be?

Regarding the last bullet point on homogenous coordinates, I tend to use float4 over float as much as I can. I find it clearer especially when vectors and positions are stored in textures. By doing so, you minimize the number of .xyz that suffixes your variable when doing float4->float3 or the float4( vec, 0 or 1) when you do float3->float4 ... So basically, I pull the needed data from the shader inpu and textures info float4s and then try to stick on working with those and not mixing types too much. I still need to investigate the performance implications but my early tests tend to show that the shader compiler is really good at optimizing all that.

I'll probably post separately about lots of 3D maths quirks so I'll have that reference material somewhere in a reliable place.

-m

Work in progress

After posting the previous ticket, I started hacking my previous forward renderer in order to implement a first draft of the new rendering loop. I generally think that even when you worked through the theory, you always end up experiencing a lot of small issues that arises while you implement you idea. That's why I wanted to quickly get my hands dirty and face the actual practical problems.That's how I learn :)

Here is a screenshot of what I have so far:

The scene features a character in bind pose and a simgle point light, placed in front of its waist. The main view is still empty as the Material Pass is not done yet, but at the bottom, you can see some debug output of GBuffers 0, 1 and 2, storing depth, normals and light preview (ie. diffuse + diffuse*specular^8) respectively.

Following is a sort of log going through various bugs/issues I had to solve. I extensively used PIX for debugging the shaders. It was very unstable because of bugs in my rendering code. Switching to the reference rasterizer allowed me to fix those bugs. In turn, the stability of PIX has improved significantly, to a point where it is really reliable.

Default values in GBuffers:
GBuffer 0 stores the linear distance to camera for each pixel. This distance will be used in lighting calculations. In order to avoid any artifact, pixel that are part of the void need to have a distance ideally equal to positive infinity. For that, and because this rendertarget is a R32F buffer, I clear it to 0x7FFFFFFF, which according to the IEEE standard is the greatest 32bits positive float value.

GBuffer1 stores the normal vector for each pixel, directly as float values. The ovious default value for this buffer is 0, so the normal will be initialized to a null vector for all the pixel that were not rendered during the Geometry Pass

Reconstructing the position from the Depth value:
I struggled quite a bit to get it right, only to realize that I had a blatant bug elswhere in one of the shaders ... That's where PIX come in handy!.. The main idea is to reconstruct the position using the depth and a ray vector that originates from the camera and passes through the current pixel. This ray can be deduced from the camera frustum corners. For that, I compute in the main app the vector that goes from the camera to the top right corner of the camera frustum. As I want this vector in view-space, it has coordinates looking like

cFrustumRayInfo = Vector3( dx, dy, 1 ) * FarDistance;

Then, in the Vertex shader, I use this vector (passed as a constant) and the clip-space position of the current pixel (which is always going to be in the [-1,1] range on both axis) to compute the ray that originates from the camera and passes through this pixel

output.vRay = float3( dx * clipspace.x, dy * clipspace.y, FarDistance );

This gets output to the pixel shader and thus interpolated per-pixel. In there, I sample the depth from the GBuffer0 filled in the Geometry Pass and I can reconstruct the position by doing

float4 vPosition = float4( depth * in.vRay, 1 );

I think that it is one of the best way to do t because that will either if you write a fullscreen quad or just portions of the screen. And depending the type of lights you're dealing with, you surely want to do both, as writing a fullscreen quad for a point light with a radius of 10 pixel is a bit of a waste, isn't it?


That's it, more stuff when'I ll be able to work further on :)

-m

Light PrePass Renderer

Over the past few weeks, I've been experimenting with various ideas for the renderer of my pet engine. I set my mind on the Light PrePass first described by Wolfgang Engel (http://diaryofagraphicsprogrammer.blogspot.com/2008/03/light-pre-pass-renderer.html) because that methods looks really promising. And then I also had the opportunity to discuss that with a friend at work who is really knowledgable about all that and that finished to convince me :)

As I progress through the implementation of the renderer in my engine, I would like to keep a log of issues and solutions I came across. Mainly becausethere is a fair amount of documentation on the principle itself but not that many about the gritty details of the implementation - or at least, that doesn't cover all my questions so far, maybe because I'm not that experienced with deferred rendering and so on.

What follows is a breakdown of thing that are already implemented with some details about implementation choices and issues.

Render Passes:

The main idea is to have 3 (or 4) rendering passes:

1. (Optionally,) A ZPrePass where objects are rendered in the ZBuffer only, without color information output thus enabling this pass to run twice as fast.
2. A Geometry Pass, where GBuffers are filled with information like the position and normal vector of the geometry for each pixel
3. A Light Pass, where light information for each pixel is calculated using the GBuffers from the Geometry pass
4. A Material Pass where information from the 2 former passes plus per-object material parameters can be combined for generating the final color value for each pixel

GBuffers:

So the first thing to think about is, as stated in many documentation, how to organize output data into GBuffers so we use as less RenderTarget memory as possible without sacrificing flexibility and image quality.

Here is what I plan to use:

 Buffer        | Format        |     8 bits     |    8 bits     |    8 bits     |     8 bits       |
---------------+---------------+----------------+---------------+---------------+------------------+
 Depth/Stencil | D24S8         |                 Depth                          |    Stencil       |
 GBuffer 0     | R32F          |                    Linear Depth (View Space)                      |
 GBuffer 1     | G16R16F       |           VSNormal.X           |           VSNormal.Y             |
 GBuffer 2     | A8R8G8B8      |    Diffuse.R   |   Diffuse.G   |   Diffuse.B   | Specular Factor  |
 Back Buffer   | A8R8G8B8      |       R        |      G        |      B        |       A          |

Those buffers will be rendered as follows:

ZPrePass: Depth is output to the DepthStencil buffer. Nothing special here, apart from turning off the color output, to enable doubling the speed of this pass. This is done by calling

m_pDevice->SetPixelShader( NULL ); 

m_pDevice->SetRenderState( D3DRS_COLORWRITEENABLE, 0 ); // restore it by passing 0x0000000F

Geometry Pass: GBuffers 0 and 1 will be filled during the Geometry Pass. As suggested in many papers, instead of storing the position, I will store the depth with a full 32bits precision and then reconstruct the position from the depth and the view/proj matrices. For the normal vector, I only store 2 of the 3 channels and reconstruct it. Also I know that many techniques for packing/unpacking normals exists and have been discussed but I'll dig into that later on.

Light Pass: GBuffer2 will receive lighting information generated during the Light Pass. During this pass, the diffuse contribution of each light is simply accumulated. For the specular factor, as it is commonly done, I choose to discard color information and only store a modulation factor that will amplify the diffuse color. This can be inaccurate in some situations where the scene contains multiple lights with different colors and intensity but I'm happy to sacrifice that over memory usage. One thing I am not too sure of right now if the format I should use for that. If I output these values into a A8R8G8B8 buffer, I believe that these numbers will be clamped in the range [0, 1], preventing me from doing the lighting in HDR. I could use some trick and always normalize it in the [0,2] range a-la Unreal to fake the HDR but I'd really like to have a proper HRD [0, +inf [ range. I'll have to dig into that a bit more.

Optionally, I could also generate a Velocity buffer during the Geometry Pass, in order to be able to make a nice perpixel motion blur as a post-process. I'll leave that for now and will come back to it later.

Wrap up:

Right now, this is as far as I went. I need to do some ground work in my engine before I can start writing code and shaders for all that but I think I have enough info to start working on a draft implementation.

I still need to clear lots of things out, here's a list of questions I'll have to answer.

• Which format for the Light Buffer if I want true HDR?
• Depth in linear space instead of homogenous space?How does the hardware do the 'divide by w' and can this be an issue?
• Effect of texture filtering when using these buffers?
• Better way of storing normals?
• Gamma space vs Linear space?
• Is there something better than a traditional forward renderer for translucent geometry?
• How to incorporate shadows in all that?

That's all for today !

-m