OpenGL 3.3 - Nature

A few months ago I’ve presented an object culling mechanism that I’ve named Instance Cloud Reduction (ICR) in the article Instance culling using geometry shaders. The technique targets the first generation of OpenGL 3 capable cards and takes advantage of geometry shaders’ capability to reduce the emitted geometry amount in order to get to a fully GPU accelerated algorithm that performs view frustum culling on instanced geometry without the need of OpenCL or any other GPU compute API. After the culling step the reduced set of instance data is fed to the drawing pass in the form of a texture buffers. In this article I will present an improved version of the algorithm that exploits the use of instanced arrays introduced lately in OpenGL 3.3 to further optimize it.

Lets recap the basics of the algorithm before I present the improved technique. The geometry shaders have a very nice feature that they cannot just emit a modified version of the input geometry but can also alter the number of emitted primitives compared to the number of received ones. This is a both-way ability what means that we cannot just increase but also decrease the number of primitives. That is what the technique takes advantage.

In the first pass we feed a simple vertex shader – geometry shader pair with the instance data of the geometries as they’ve been the data of point primitives. The vertex shader then checks whether the actual instance is inside the view frustum or not and sends the result to the geometry shader. If the result is yes then the geometry shader outputs the instance data otherwise discards it. The primitives emitted by the geometry shaders are captured then using transform feedback into a buffer object. Also a query object is needed in order to be able to get the amount of instances that passed the view frustum culling. In the drawing pass we use the result of the query to decide how many instances we have to draw and the captured feedback buffer is used as instance data.

Instance Cloud Reduction - Combined view of Pass 1 + Pass 2

This is a very brief description of the culling mechanism so for a complete specification please read the original article.

Motivation

While Instance Cloud Reduction is a quite robust technique that can severely simplify and speed up the rendering of high amount of instanced geometry its performance is also limited due to some hardware and API restrictions. The most important ones are the following:

• Needs an extra rendering pass to perform the culling.
• Requires the usage of asynchronous queries to determine the number of visible instances.
• Uses texture fetching in the vertex shader of the actual drawing pass.

The first mentioned drawback means that more draw commands are required that use the output of the first pass as input. This and the second disadvantage may cause stalls due to the fact that the CPU has to wait for the data to be ready before issuing the second pass thus the GPU is not used effectively.

What this improvement tries to solve is the third problem. Texture fetching itself is quite fast in the latest generation of hardware, however it causes some slowdowns anyway due to the latency introduced by texture fetches even though GPUs use some latency hiding techniques.

Instanced arrays provide us a way to replace texture fetching with vertex fetching that is usually done by different hardware element that works synchronously with the execution of vertex shaders. I’ve expected quite a reasonable speedup by taking advantage of instanced arrays, however we will see that actual results were far from my initial expectations.

Implementation

Traditional vertex fetching happens in a way that one element is fetched from each enabled input attribute buffer and the vertex shader is issued with these values. One element in a vertex attribute buffer can mean up to four floating point or integer values and for each execution of the vertex shader one set of these elements is used. There is an internal counter that is increased after each fetch and the next vertex attribute fetch will use this counter as an index into the buffer object.

While this mechanism is satisfactory for the most attributes of a vertex, it is not practical for instance data as such data belongs to an instance rather than a vertex. In order to source instance data from vertex attributes in case of traditional vertex fetching, high amount of redundant storage is required in order to get the same information for all the vertices belonging to a particular instance. This is not just waste of memory but also waste of bandwidth and it also defeats the goal of Instance Cloud Reduction.

Compared to traditional vertex fetching, instanced arrays provide a way to increase the internal counter used as the index into the vertex attribute buffer in a different way, in particular one can set the frequency of increase using a vertex attribute divisor that specifies after how many instances the counter shall be increased. This is a per-attribute property and by setting it to one we end up with exactly what we need: one vertex fetch per instance.

This means that actually we need just a very minor change compared to the original technique, more precisely we replace our texture buffer with a vertex attribute buffer that has a divisor of one and use it as the source of instance data in the vertex shader of the drawing pass.

Execution results

As we are not talking about a new technique but just an optimized implementation of the same method, the best way to evaluate it is by comparing the performance of the new version with the original one.

As I’ve mentioned earlier, I expected a reasonable performance increase by replacing texture fetches with vertex fetches, in practice the difference was not so significant. However, the performance difference between the two implementation can heavily depend on the underlying hardware implementation so various cards from various vendors and GPU generations can show more diverging behavior. In fact even driver versions may have an effect on the results.

Performance comparison of the old implementation and the presented one on an AMD Radeon HD5770. Scale is in frames per second (higher value is better).

Due to lack of hardware to use for testing, I’ve checked only with one card, namely a Radeon HD5770 with Catalyst 10.6 drivers. I noticed roughly a 10% speedup as the the new version of the Nature demo showed 100 FPS compared to the 90 FPS observed with the old implementation.

Even though this was not exactly the outcome I’ve expected from the new implementation, maybe the assumption is still valid for older generation of GPUs or for NVIDIA cards. I suspect so because for Shader Model 4.0 cards the hardware implementation of the texture fetching unit and the vertex fetching unit was most probably more differentiated than that of the latest GPUs. Also my guess is that on NVIDIA cards the difference is maybe higher as the vertex fetching hardware in SM 4.0 GeForce cards is less flexible than that of AMD’s taking in consideration that the first HD series Radeons already had some form of tessellation functionality that requires more freedom from the vertex pushing hardware.

In order to get a better picture about how effective the presented optimization is, I would like to ask all the visitors of this post to try the two releases and send me feedback about it.

Conclusion

We’ve seen that how easy it was to take advantage of instanced arrays in an existing implementation of the ICR technique and how does it perform on the latest generation of GPUs compared to the previous version. While this small addition provides some benefits, it also comes at a cost and we have to talk about that as well.

Advantages:

• Eliminates the need for texture fetching in the vertex shader thus improving performance.
• Does not compromise the goal and the implementation architecture of the original method.
• Frees up one texture unit that was previously reserved for the texture buffer containing the instance data.

Disadvantages:

• Requires OpenGL 3.3 or the GL_ARB_instanced_arrays extension in addition to the OpenGL 3.2 features.
• We have to possibly sacrifice multiple vertex input attributes to feed the instance data to the shaders.

Most of the mentioned benefits and drawbacks are self-explanatory, however I would like to say a few words about the last mentioned one…

For the purpose of showcase I used a simple translation factor as instance data that means a single vector of floats. In real life situation one may need more complex transformation data that can only be stored in the matrix. While in the demo the feeding of instance data consumed only one vertex attribute slot, in case of a full transformation matrix it would require four of them (not to mention other possible instance attributes). As the maximum number of input attributes is severely limited, usually to 16, the application of the optimization is restricted to situations when all the vertex and instance attributes fit into this limit.

In case of the original implementation, where a texture buffer was used as input, this did not cause any problem as the vertex shader is free to fetch any number of texels from that (still, performance can be a concern in this case). In order to help situations when input attribute slots are at a premium, in real life scenarios it is recommended to use quaternions instead of transformation matrices as they consume two times less attribute resources. Actually this can be a general recommendation as using quaternions decreases the bandwidth requirements of the instance data fetch thus increasing performance even in situations when there are enough input attribute slots available.

In order to ease the performance comparison for you, you can find download links for both versions of the Nature demo.

Old version binary release

Platform: Windows
Dependency: OpenGL 3.2 capable graphics driver
Download link: nature12_win32.zip (3.58MB)
Comments: This version does NOT include the optimization presented in this article.

Old version source code

Language: C++
Platform: cross-platform
Dependency: GLEW, SFML, GLM
Download link: nature12_src.zip (12.6KB)
Comments: This version does NOT include the optimization presented in this article.

New version binary release

Platform: Windows
Dependency: OpenGL 3.3 capable graphics driver
Download link: nature20_win32.zip (3.58MB)
Comments: This version includes the optimization presented in this article.

New version source code

Language: C++
Platform: cross-platform
Dependency: GLEW, SFML, GLM
Download link: nature20_src.zip (12.8KB)
Comments: This version includes the optimization presented in this article.

Everybody who used to make OpenGL applications, whether it be a simple triangle-of-death demo or a comprehensive rendering engine at some point needs to use extensions or later OpenGL versions. Usually many people start this by creating their own initializer library that loads the required entry points from the OpenGL library by hand. What is sure is that at some point everybody realizes that this process is just a waste of time and starts to look for an extension loading library out there. This is the obvious solution as it makes no sense to reinvent the wheel all the time. However, after using a particular one from the repertoire of these libraries one will face the problem that they are not that nice as they seemed before. In this article I will talk about some of these libraries and some of my thoughts about them.

OpenGL is evolving in a more and more fast manner nowadays and it is crucial to have an extension library that serves your needs and is up-to-date enough so you can easily adopt the latest features of the API. The sad truth is that this is not the case, at least there are some pitfalls that can cause you a lot of headaches when relying on these libraries.

This week, I planned to create a new version of my Nature demo first presented in my article Instance culling using geometry shaders that adopts the latest features of OpenGL 3.3, especially concentrating on how GL_ARB_instanced_arrays can improve the throughput of the technique. I know that I would be able to do this without actual OpenGL 3.3 support from the extension library by using the extension itself rather than the core functions, but how it would like if I publish a demo stating it’s OpenGL 3.3 but I use internally OpenGL 3.2 with extensions? That would be just too airy.

As you may know, I used GLEW in my previous demo as the extension library of choice, but I had some difficulties at that time as well. This time I got pissed off much easier as the bad memories put their mark on my preconception. I will talk about the reasons behind this later. First I would like to clarify the subject behind this article.

As I had several bad experiences with various libraries, especially this week, I thought it would be nice to write a summary of the possibilities have regarding to the topic and what are the advantages and disadvantages of each, at least based on my experiences. The libraries I will talk about are: GLee, GLEW and GLLoader. I don’t want to blame the people who developed these libraries so excuse me if I will be sometimes too harsh as I should be already satisfied with the fact that at least I have the opportunity to use such open source tools instead of reinventing the wheel myself, but if you understand the background of my feelings you may easily accept why I’m unsatisfied…

A few years ago I was developing all my hobby projects in Delphi/Free Pascal due to the great facilities and the nicer coding style that the language provides. I changed my mind in the recent past and switched to C/C++ because of the greater developer community behind the languages and to eliminate the need to develop supporting libraries for myself that already exist for other languages. So my assumption was that if I do this shift I will have far less problems with gathering the modules that will do the secondary stuff needed by my projects. Unfortunately, I soon became very disappointed due to the quality properties of those third-party tools that I thought can be the holy grail for my hobby projects. Of course, I had many good experiences, there is also one that I shared with you. But enough from the doublespeak, let’s see what we have…

GLee (GL Easy Extension library)

I don’t have too much experience with this library as the last time I used it was years ago. Because of this, it is hard to say too much about it but I think one simple facts are enough to justify why this library should not be the primary choice for enthusiasts: it is pretty outdated as it supports only OpenGL 3.0 that has been released quite a long time ago. It even seems to me that it is no longer developed. It is written as hard-coded source that is not even an easy to maintain library, most probably that is the reason behind the disappearance of it.

Beside that, there are also some good things about this one. Namely it comes with a source code that can be easily integrated and compiled without the need of any third party software and I seriously consider this as an advantage, later you will understand why. Also, it comes with BSD license that allows enough freedom for almost any project.

Anyway, even though this library is simple and good for many applications, the fact that it’s outdated and not really maintained is a warning sign that every developer starting to use it must consider.

GLEW (OpenGL Extension Wrangler library)

As I mentioned earlier, this is the library I used for my latest demos so I have up-to-date information about it. The one thing I like in it the most is that there is an excellent design idea behind it that, in theory, would make it the most superior library for the purpose. I intensionally used the expression “in theory”, I will explain it soon. The library comes with BSD or MIT license that is also a nice thing.

GLEW has a very nice build system behind it that can automatically download extension specifications from the OpenGL Extension Registry and generate the library using that information. Unfortunately, this build system is the one that can make you much headache if you would like to use it under Windows as it relies on many POSIX tools.

According to the homepage, GLEW shall be compiled very easy even under Windows with cygwin. I have only two problems with this:

1. I use Windows and I use MinGW for compilation and even with the tool-set of MSYS the building of the library renders almost impossible.
2. As a last resort, I also tried cygwin but had similar results with it as well.

Actually this wouldn’t be a problem on its own if the project maintainers would release Windows binary versions at least in case of every update of the OpenGL core specification or, at least, they would generate the source files in such cases as for compiling the sources themselves there is already a nice Makefile and a Visual Studio project file as well. Unfortunately, this is not the case. I don’t remember when they’ve released the binaries for OpenGL 3.2 but I’m sure it was far after the specification release. Anyway, the best way to solve the problem would be to create a build system that works also on Windows as it makes the library quite inconvenient for Windows developers.

GLLoader (part of Klay Game Engine)

I heard about this one just in the recent past, after the release of OpenGL 3.3 and 4.0. This was the first extension library stating that it’s OpenGL 4.0 capable. Well, there is not too much to complain about this tool, only just a few things:

• By default, it comes with a dynamic library project file and I don’t really like to supply a DLL with my demos just for extension loading.
• There are some mistakes in the code (at least I found one related to glMapBufferRange function).

Also, unfortunately, it comes with GPL licensing that is too restrictive for many use cases.

Conclusion

There are actually several possibilities if one has to choose an OpenGL extension library, but unfortunately each has its drawbacks. GLEW would be obviously the most superior solution if there wouldn’t be problems regarding to its build system. I would even consider correcting the problems during compilation (what usually occur due to the code generation as it does not handle all special cases, like it happened earlier with WGL_ARB_create_context that had multiple versions of it).GLee seems to be a definite no, but GLLoader is maybe a factor to consider.

We will see how each project will progress and whether they will care about the minor but annoying problems with their products (I already created some bug/support report regarding to the met issues mentioned). Anyway, currently it seems that this area still has unfilled gaps so one can easily drop in with its own library and capture the attention of developers who are seeking for third-party supplementary tools. My hope is that the developers of GLEW will make a step and resolve the issues thus creating a real plug & play library that everybody can rely upon.

Nowadays comprehensive testing is a must for any software product. However, it isn’t such a general rule when it comes to graphics applications. Many developers face difficulties when they have to test their rendering codes. Manual tests and visual feedback is sometimes satisfactory but if one would like to have automated regression tests usual approaches seem to fail. Even if at first sight unit testing of rendering code doesn’t look really straightforward, in fact it is. OpenGL is not an exception from this rule as well. In this article I would like to briefly present a few methods how to unit test OpenGL rendering code and also present my choice and the reasons behind the decision.

There are several ways how to create automated test cases for rendering code. To present the different approaches we first have to select a small portion of our rendering code to demonstrate the differences of each technique, mentioning the strengths and weaknesses of them.

Before going any further, we have to lay down our requirements against a good OpenGL unit testing environment:

• Verifies results – This is the most basic requirement for any testing framework. We have to have the ability to check whether the rendering code executed by the module is valid and works as expected.
• Productive – The usage and maintenance of the framework shall require minimal effort. Many times unit testing is attacked because it requires additional code writing. While this is generally true, a nice unit testing environment can be kept very simple yet flexible. An OpenGL testing environment shouldn’t be different.
• Fast – This is a general requirement for any unit testing environment especially when combined with a continuous integration framework. We want our test results as fast as possible as long feedback cycles severely slow down the development process.
• Standalone – Does not require complex setup or environmental support in order to be executed. This is a general requirement when we deal with unit testing as if the code is tightly coupled by any of the surroundings then both development and maintenance costs increase.
• Compatible – Does not require any special hardware so it can be tested on a machine that wouldn’t necessarily be suitable for manually testing the actual product. This is especially important when the target hardware is some type of embedded platform. It is also important to ensure that it will work on hardware provided by different vendors. In one word, it should comply to the standard, not to driver implementations.
• Cross-platform – Does not rely on the services of a particular operating system or platform, instead it can be executed on any machine as usually all unit tests. Of course, this restriction can be relaxed depending on actual use case scenarios.

Now that we know what we would like to achieve, we can continue with a sample use case. Lets say we would like to create an OpenGL 3.2 based rendering engine. One of the first things that we would write is a class (or set of classes) that will help us handling OpenGL buffer objects as it seems to be one of the main building blocks of such a system. As a very basic example, our first version of the buffer handling class will act simply as a wrapper for buffer objects having the following interface:

class Buffer {
public:
    Buffer();
    virtual ~Buffer();
};

As it can be seen for now we just require that our class to handle the creation and deletion of a buffer object. Obviously, our test has to check that the constructor successfully creates a buffer by calling glGenBuffers and the destructor deletes that by calling glDeleteBuffers with proper arguments. Now lets see what possibilities we have to test OpenGL rendering code and whether it conforms to our requirements and is able to test our simple module.

Checking rendered image

The most naive solution for creating automated tests for rendering code is to actually execute the OpenGL commands and check whether the rendering happened as expected. This can be done by comparing reference rendering results to the actual ones. This approach has the benefit that we actually verify the concrete behavior but lets see how it looks like when we check against our previously laid down requirements:

• Verifies results – Partially fulfilled. We check against the correct behavior, however, the ability to reproduce the actual same image is often difficult if not impossible due to different relaxations regarding to precision in both the standard and driver implementations. In order to have reproducible results the testing environment shall also provide some mechanisms to allow slight differences.
• Productive – Not met. It can be quite expensive to create an assertion system. Also, the production of reference data can be quite time consuming.
• Fast – Not met. Even if the checkers are highly optimized components of the framework, it wouldn’t fit into the time-frame of unit test cycles to execute possibly thousands of test cases that require complete verification of the produced image.
• Standalone – Not met. We have to setup a complete rendering environment in order to test even the simplest rendering code. Also, it relies on the assumption that the rendering code actually produces some image. As we can see in our buffer handling example, this is not always the case.
• Compatible – Not met. We need a testing machine that has the hardware capabilities to execute the rendering code and produce the required image.
• Cross-platform – Partially fulfilled. If our rendering code is cross-platform then it is possible to test it on any of the supported platforms. However, this makes the assertion system even more complicated as it also has to support the target platforms. Also, driver implementations may vary even further when dealing with different operating systems.

As we can see, even if this version is quite natural way of thinking for anybody it’s simply impractical and not feasible for actual use. To be able to find a good solution we must look deeper into what unit testing exactly is as the presented solution has nothing to do with it. In order to be able to do real unit testing we have to eliminate the dependency on OpenGL driver implementations and strictly concentrating on the module under test.

Fake OpenGL driver

The second presented solution is to create a layer between the code under testing and the actual OpenGL driver implementation. This can be easily achieved by creating a fake driver, as an example a dynamic library called opengl32.dll in case of Windows. This additional layer would do nothing else than just recording and checking whether the required API calls happened as expected. Providing an interface towards the testing environment that can be used to request the informations needed to make a verdict about the successfulness of the test case.

Beside that this version accommodates much more to the idea behind unit testing it also has the benefit that it is acting as a totally independent layer and does not directly disturb the development of the actual code. Still, if we go back to our checklist we have some issues that raise some concerns regarding to the applicability of this approach:

• Verifies results – Partially fulfilled. It is up to the implementation of the new layer whether it provides the required facilities to properly check the behavior of our tested code. Nevertheless, it also highly depends on the implementation on how we define correct behavior and the responsibility of the library.
• Productive – Partially fulfilled. Now we have a separate module that helps us in the testing. This may introduce some additional maintenance work but, of course, this depends on how intelligently is the library actually implemented.
• Fast – Mostly resolved. We do not have expensive assertions, however, as we have a quite restricted interface between our testing environment and the new layer we most probably met situations when we have to make trade-offs between speed and flexibility.
• Standalone – Resolved. We have a totally independent module that is responsible to simulate the surrounding environment of the code under testing as it should be when doing unit test. However, the question arises whether we would like this layer to be that separated from the testing code.
• Compatible – Resolved. There is no dependency on dedicated graphics hardware or any other piece of metal. In case of a robust driver simulation layer we can test our code on whatever platform we prefer.
• Cross-platform – Resolved. As previously mentioned, if the additional layer is well designed, there should be no problems regarding to this issue.

Now we have a resolution that can be seriously taken into consideration as a good way to test rendering code. It can also be simply applied to test our buffer handling code as well. Also, as it is a totally standalone software element it is also very portable so it is easy to reuse between projects written in different programming languages and for different platforms.

Still, there is one thing that may need further investigation. Most probably for the other portions of our production code we already use some kind of mocking mechanisms for our unit testing. Having an additional interface type to handle the OpenGL related mocking (as the presented fake driver approach is nothing more than a mock library for OpenGL) may reduce the productivity of our developers. Also, it can make the testing code less uniform so introducing a slight maintenance penalty. At least for comparison, we should try to integrate the OpenGL mocking into our existing mocking facilities.

API mocks

All the people who seriously do unit testing use some mocking techniques to eliminate dependency on any external software element like databases, network or another code element. Why should the OpenGL API be different?

As I already written about that I use GoogleMock to test my C++ code. Lets see how this mocking framework is capable for removing OpenGL related dependencies. By default, GoogleMock does support only class mocks, however it is fairly straightforward to mock out OpenGL API functions as well. As an example, our buffer handling class needs at least a mock for the glGenBuffers and glDeleteBuffers API functions. These mocks can be very easily created using GoogleMock as part of a class in the following way:

class CGLMock {
public:
    MOCK_METHOD2( GenBuffers, void(GLsizei n, GLuint* buffers) );
    MOCK_METHOD2( DeleteBuffers, void(GLsizei n, GLuint* buffers) );
};
CGLMock GLMock;

This, however is not enough to replace the already existing real API function pointers with the fake ones. I did this with a nasty little trick by taking advantage of the C preprocessor:

#undef glGenBuffers
#define glGenBuffers                  GLMock.GenBuffers
#undef glDeleteBuffers
#define glDeleteBuffers               GLMock.DeleteBuffers

The #undef is needed because I use GLEW for accessing OpenGL API functions and it uses macros for the API function names as well.

All these are put into a file that can be called like glmock.h. In order to force the production code to use these definitions when trying to access the API inside a test case we have to create a wrapper header called something like opengl.h that will include original headers in case of normal build and include the mock library in case of unit test build. This is kind of a workaround but it works quite well in practice.

In theory, this trick can be applied in case of any mocking framework. As a result, from now we can write a very simple test case to check the creation and deletion of our buffer object as easily as the following few lines of code:

TEST(BufferTest, CreationAndDestruction) {
    EXPECT_CALL(GLMock, GenBuffers(1,_))
        .WillOnce(SetArgumentPointee<1>(13));
    Buffer* buffer = new Buffer;
    EXPECT_CALL(GLMock, DeleteBuffers(1,Pointee(13)));
    delete buffer;
}

I would not like to go into the details related to the interface of GoogleMock. In one word, the test case above checks whether the constructor calls glGenBuffers with a number of 1 for the requested number of buffer objects and returns a buffer ID in the pointer argument, and at the end it checks if glDeleteBuffers was called with the buffer ID value got at creation.

It is maybe a matter of taste whether the second or this third solution is more attractive for you. My choice was this last solution because I didn’t want to develop an separate library and also was afraid of messing up my test code with different syntactical representations of mocks. Finally, lets sum up the achievements of this last version:

• Verifies results – Fulfilled. An existing mocking framework is used for emulating the OpenGL API thus we have all the facilities required for the proper checking of the API calls.
• Productive – Fulfilled. Again, we don’t have to deal with writing an own mocking mechanisms as we have everything out of the box. We can also incrementally extend our mock library on-the-fly while editing the test cases and the production code.
• Fast – Resolved. Our rendering related unit test cases should be as fast as any other test codes as they are indifferent, just the purposes are dissimilar.
• Standalone – Mostly resolved. The mocking library is independent, however, as we’ve seen, the introduction may require some nasty tricks in order to inject foreign code into the production code.
• Compatible – Resolved. From this point of view, this approach behaves the same as the previous version.
• Cross-platform – Resolved. Again, the same like in the previous case, maybe even a bit easier to make it portable.

Conclusion

We’ve seen a few ways how we can extend our testing environment in order to support the verification of rendering code. We’ve also seen that the range varies from techniques that provide high level methods suitable especially for functional testing, until very low level methods that tightly integrate in the mocking methodology of unit testing. These, of course, do not replace traditional testing methods rather they extend it in order to find problems in the early phases of software development.

I also tried to present a very basic example of production code that needs such a facility in order to be tested, as well as a sample test case written using GoogleMocks applying the last presented technique.

While writing this article I got the idea that it would be nice to have a complete and general framework for OpenGL testing. If there is interest for it, maybe I’ll allocate some time to write one. I’m also interested which approach is the most attractive for you, especially if you have some concrete experience with any of these or with some other technique.

OpenGL 3.2 - Nature

Since the appearance of Shader Model 4.0 people wonder how to take advantage of the newly introduced programmable pipeline stage. The most important feature enabled by geometry shaders is that one can change the amount of emitted primitives inside the pipeline. The first thing that a naive developer would try to do with it is geometry tesselation. However, the new shader performs very bad when used for tesselation in a real life scenario even though there are demos show casting this possibility. If we take a closer look at the new feature we observe that the most revolutionary in it is not that it can raise the number of emitted primitives but that it can discard them. This article would like to present a rendering technique that takes advantage of this aspect of geometry shaders to enable the GPU accelerated culling of higher order primitives.

Geometry shaders can be used for many different advanced rendering techniques that were impossible before the introduction of this flexible programmable shader stage. In this article I would like to present one use case that for me seemed to be one of the most practical application of primitive manipulation possibilities introduced by geometry shaders. As I haven’t seen any whitepaper talking specifically about this particular technique, even if some of them inherently used it, I would dare name the technique myself as Instance Cloud Reduction. I will also present a demo program that shows how to take advantage of the technique in a heavy workload situation.

The idea itself was inspired by AMD’s  tech demo for the Radeon 4800 series cards called March of the Froblins. An almost identical technique presented in this article is used in the mentioned demo for the culling of large amount of animated creatures against the view frustum. Also a somewhat similar technique is used in NVIDIA’s Skinned Instancing demo for determining LOD instance sets. Unfortunately, both demos are for DirectX only and, as far as I can tell, there is no OpenGL demo showing any of the aforementioned rendering techniques.

Motivation

Nowadays, as the computational capabilities of GPUs is growing in a much faster pace than that of CPUs, graphics developers meet more and more optimization problems related to CPU bound applications. More and more focus is on minimizing the number of driver invocations, actually that’s what motivated the restructuring of the two most commonly used graphics APIs. As a result we have now DirectX 10+ and OpenGL 3+. However, even if the introduction of geometry instancing, texture arrays and local memory buffer storage for the most important inputs of the rendering, there is still need for wise decisions from graphics programmers to take full advantage of the horsepower coming with the latest GPUs.

Earlier graphics applications strongly relied on CPU based culling techniques, whether it be the usage of the quite outdated BSPs or the more generic and still heavily applied hierarchical culling techniques. We’ve already reached the point that sometimes even the most efficient CPU based culling techniques seem to be too expensive and usually introduce the small batch problem. Instanced rendering is not an exception.

The applicability of geometry instancing is strongly limited by several factors. One of the most important ones is the culling of instanced geometries. One may choose to cull these objects in the same fashion as others, using the CPU, but that usually breaks the batch and maybe we loose the benefits of geometry instancing. It is more and more imminent to have a GPU based alternative. Without CPU based culling, by sending the whole bunch of instances down the graphics pipeline may choke our vertex processor in case we have high poly geometries and quite large amount of instances of it.

The rendering technique presented in this article will try to achieve this goal. We will use a multi-pass technique that in the first pass culls the object instances against the view frustum using the GPU and in the second pass renders only those instances that are likely to be visible in the final scene. This way we can severely reduce the amount of vertex data sent through the graphics pipeline.

Implementation

For some people it might seem that the promise for such a technique is simply too naive and is most probably relying on very exotic OpenGL features, heavy misuse of some basic features or need of data conversions during the frame rendering. Wondrously, this is not the case as we have all we need in OpenGL 3.2 to implement the object culling method sketched above. All we need are the followings:

• instanced rendering (core since OpenGL 3.1)
• geometry shaders (core since OpenGL 3.2)
• transform feedback (core since OpenGL 3.0)
• uniform or texture buffers (core since OpenGL 3.1)

The method itself is a multi-pass rendering technique, however, unlike other multi-pass rendering techniques it does not produce any fragments in the first pass, instead the first pass does the view frustum culling and processes data entirely only inside buffer objects.

Culling pass

In the first pass we will feed the graphics pipeline with information about the instances that are needed to perform the view frustum culling. For this we need two inputs for the executed shaders in order to be able to perform the required calculations:

1. Instance transformation data (whether it be a simple transformation matrix or quaternions or whatever) -- This preferably comes from one or more buffer objects that are bound as vertex buffers to the context.
2. Object extents information -- Beside the instance positions we have to know the extents of an instance in order to perform correct culling. This can be either a single float representing the object radius if we choose to use bounding spheres for the culling or a three-dimensional extent vector if we would like to use bounding boxes.

Using these as input we can feed in the instance transformation data as attributes of point primitives to our culling shader. The culling shader is composed of a vertex and a geometry shader. In a typical setup the role of each is the following: the vertex shader determines whether the actual object instance’s bounding volume is inside the view frustum and sends a flag about the culling to the geometry shader, that will emit the instance data to the destination buffer if the flag says that the instance is likely to be visible or does not emit anything if it is determined that the object instance is out of view.

Next, transform feedback is used to capture the primitives emitted by the geometry shader into another buffer object that will be used in the actual rendering pass to source instance transformation data. Beside this, we also need to have an asynchronous query to determine the number of primitives generated to know how many instances of the object do we actually need to render. The following figure shows the workflow of the first pass:

Instance Cloud Reduction - Pass 1: Culling

The actual geometry shader implementation needed to perform the actual culling based on the view frustum check performed by the vertex shader should look like the following chunk:

#version 150 core

layout(points) in;
layout(points, max_vertices = 1) out;

in vec4 OrigPosition[1];
flat in int objectVisible[1];

out vec4 CulledPosition;

void main() {

	/* only emit primitive if the object is visible */
	if ( objectVisible[0] == 1 )
	{
		CulledPosition = OrigPosition[0];
		EmitVertex();
		EndPrimitive();
	}
}

In this example we used only simply a four-component position vector for the instance transformation data but the technique works well for transformation matrices and quaternions as well.

One more thing is that beside that we set up transform feedback in a way that we feed our buffer object dedicated for the culled instance data and we also started an asynchronous query to be able to determine the number of primitives written into the buffer object, it is also useful to turn of rasterization as we wouldn’t like to produce any fragments as a result of the first pass.

Rendering pass

In the second pass there is nothing special to do. Simply use whatever rendering setup you would like to use. The only things that need to be changed in this step compared to your already existing rendering path is that the instance data for the rendering must be sourced from the generated culled instance data buffer and, as a result, the number of instances passed for the instanced drawing functions shall be changed in order to render only the visible instances. This number can be read from the asynchronous query’s result that we started in the first pass.

The instance data in the rendering pass can be, of course, sourced from either a uniform or a texture buffer object. This depends on the actual use case and is more clearly explained in the article Uniform Buffers VS Texture Buffers.

Important note is that when one has to deal with several instanced geometries it is recommended to do the culling phase prior to rendering any instanced primitives because of the following reasons:

• The result of the first instance cloud’s culling is more likely to be finished on the GPU so no sync issues arise from reading the asynchronous query result to determine the number of visible instances.
• Probably less state changes are needed as very different setup is required by the two passes.
• Results in tidier renderer design as culling is clearly separated from actual rendering.

Putting everything together, the application of the presented technique would result in the following workflow on the GPU:

Instance Cloud Reduction - Combined view of Pass 1 + Pass 2

Conclusion

We’ve seen that the presented advanced rendering technique is able to help in situations when we have to deal with large number of instanced geometries and how to take advantage of the latest features of graphics cards and OpenGL to perform view frustum culling calculations on the GPU. This prevents us from having to deal with complicated and expensive CPU based object culling methods that break the drawing batches, especially when dealing with dynamic objects. For ease the decision whether to incorporate this technique in your rendering engine I would like to present the advantages and disadvantages of it.

Advantages:

• Heavily reduces the amount of processed data in a naive implementation.
• No need for any space partitioning methods in the host application to handle the culling of dynamic objects.
• Can handle huge amount of instanced objects due to the enormous horsepower of today’s GPUs.
• Scales well with increased number of instances as the per-instance calculation is relatively low.
• Relies strictly on OpenGL 3.2 core features.
• No need for OpenCL capable hardware.

Disadvantages:

• Needs an extra rendering pass to perform the culling.
• Requires the usage of asynchronous queries to determine the number of visible instances.

I hope you agree with me and think about this technique as one more step towards fully GPU based scene management. If you have any remarks or improvement ideas regarding to the rendering technique itself feel free to tell me.

The Demo

As I promised, the technique presented above comes with a live demo that actually took most of my time dedicated to writing this blog in the last two weeks. The demo itself is more like a technical show cast rather than a presentation of a real-life use case scenario.

First of all, I used high polygon count models for the rendering to emphasize the amount of time the culling phase spares from the very valuable time of our GPU. In a real world application one would never do something like this. As a result, the demo is more like a benchmark than an interactive application. However, maybe on high-end graphics cards it can perform pretty well.

The demo scene consists of two object types: trees and grass blocks. The tree model is further divided into two parts as they need different textures: the tree trunk and the tree foliage. Obviously, this additional burden can be prevented by using texture arrays to avoid the need of separate draw calls to render the trunk and the foliage.

The tree trunk consists of 33138 triangles, the tree foliage has 16069 triangles and the faking-free grass block consists of 8961 triangles which I had to model myself as didn’t found any suitable model. Actually this modeling step consumed quite a reasonable amount of my time spent with the demo as I’m not an expert in this domain.As you can see, these models are not the ones that one might use in an interactive real-time application like games. However, they seemed to be very suitable for the purpose of the demonstration.

What really kicks off the boundaries of GPUs is that the demo renders 10,000 trees and 250,000 grass blocks using instancing. This ends up in more than 2.7 billion triangles in the scene. This is far more that a GPU can handle without the aid of some scene management and culling. However, we will use no scene management at all and the only culling method that we will use is the one presented in this article.

The actual results are quite promising. The view frustum culling step usually spares more than 99.9% of the GPU horsepower as the amount of actually rendered triangles after the culling step is far below 2 million triangles. This is still quite much but as we use high polygon count models and we don’t use any LOD techniques this seems reasonable.

Even if the demo scene statistics doesn’t seem like a typical use case scenario, the ease of the implementation and the compelling visual results made me pleased anyway:

www.youtube.com/watch?v=srbOFTLTe8k

On my Radeon HD2600XT I have achieved 6-7 frames per second which is acceptable taking in consideration the huge amount of geometry data still passed to the graphics card. On more recent cards I suppose it should run with good frame rates, however, due to the lack of hardware to test on, these are my only results. If anybody manages to take a better screen capture than mine above then please let me know.

Implementation details

Just to tell a few words about what techniques and tricks I’ve used during the creation of the demo here is a listing of the most important ones:

• Three models are used as mentioned previously with high instance counts with over 2.7 billion of total triangles in the scene as mentioned already.
• Three 512x512 RGBA textures are used for the models that are partially handmade, and again, I’m not a texture artist so sorry if they don’t look flawless.
• The wavefront model and TGA image loader that accompany the demo are very roughly implemented only for the demo so I would strongly encourage you not to use it to any purpose as it handles only a subset of the possibilities of the file formats.
• The vertex data from the wavefront model files is transferred in a very naive way so vertex reuse isn’t taken into account.
• The instance data consists of simple four-component vectors representing the world-space position of the instance. This seemed to be the most simple for the demonstration purposes.
• In the second pass, the instance data is sourced from a texture buffer but not really because the visible instance count exceeded the amount that would fit in a uniform buffer. I used texture buffers because for this simple demonstration they seemed to be a little bit more easy to be integrated.
• The morphing effect that simulated wind blow is done using hard-coded geometry deformation in the vertex shader. It is not physically correct but visually compelling.
• The lighting is a simple directional light using Phong’s shading and reflection model.
• Simple fog is simulated with some awkward formula that I’ve chosen after a few test runs.
• Alpha testing is achieved by using the discard operation in the fragment shader.

Driver issues

During the development of the demonstration program I’ve met several driver related problems as I’ve never used so heavily the latest OpenGL features previously. I’ve worked with Catalyst 9.12 and 10.1 but both seemed to lack of a proper GLSL compiler. Here are some of the issues I’ve met:

• When I’ve forgot to declare the varyings in the geometry shader as arrays like the standard requires then still the driver hasn’t complained about any syntax error but when tried to execute the code the program crashed.
• Except the texture sampler uniform, all other uniforms failed to work when used in the fragment shader only so I’ve put them all in the vertex shader.
• For loops seemed not to work when used inside the geometry shader, that’s why the culling itself is done in the vertex shader in the demo.

All these problems resulted in nasty tricks to make things working and ended up in awful shader code. Sorry for that. At least now it works on my configuration but pretty unsure whether it will work on other graphics card and driver combos. Please report me any success or failure when trying out the demo. Anyway, be sure to have the latest graphics drivers installed as, at least in case of AMD, OpenGL 3.2 drivers came out only at the fall of 2009.

Edit:

Thanks to the information got from Pierre Boudier from AMD I’ve updated both the source and binary releases to support the latest drivers properly. The problem was that I didn’t use attribute location binding as specified in the standard.

Also have to mention that with my new Radeon HD5770 I managed to achieve over 90 frames per second that actually show that this technique can be in fact used for games and other interactive applications.

One more thing in the end. As you know this version of the Nature demo uses a texture buffer to source instance positions. I plan to create another version that will take advantage of the instanced arrays introduced in core with OpenGL 3.4. I expect quite a reasonable speedup as that would eliminate the need for texture fetches in the vertex array by rather dedicating a vertex fetcher for the purpose thus increasing the overall performance of the technique.

Binary release

Platform: Windows
Dependency: OpenGL 3.2 capable graphics driver
Download link: nature12_win32.zip (3.58MB)
Comments: Includes the update that makes it work even with the latest drivers.

Full source code

Language: C++
Platform: cross-platform
Dependency: GLEW, SFML, GLM
Download link: nature12_src.zip (12.6KB)
Comments: Sorry for the many dependencies, however, I would recommend the mentioned libraries for everybody who is doing OpenGL development.