I’ll first present the generic steps of AdMob integration that are also the prerequisites of setting up AdMob with cocos2d-x.

Step 1 – Adding the AdMob SDK library to your project

First, you have to add GoogleAdMobAdsSdk-x.x.x.jar to your project. In Eclipse you can do so by going to Project -> Properties -> Java Build Path, and then on the Libraries tab clicking on the Add JARs… button and then selecting the jar file in the file browser to add it to your project. In my projects I like to keep the jar file inside my source tree so you probably also want to first copy the file to your project’s libs folder before adding it to the project in the Eclipse GUI.

Step 2 – Setting up your AndroidManifest.xml

The second step is to set up your Android manifest file. In order for the AdMob SDK to be able to present the ads you have to add the following activity definition under your manifest/application element in the XML:

<activity android:name="com.google.ads.AdActivity" android:configChanges="keyboard|keyboardHidden|orientation"/>

Next, you have to declare the Android permissions that are needed for the SDK to work. At a bare minimum INTERNET and ACCESS_NETWORK_STATE permissions are required. You can request the permissions by adding the following two items under the manifest element in your XML:

<uses-permission android:name="android.permission.INTERNET"/>
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE"/>

For some configurations you might need to add additional permission requirements to your application, but I’ll talk about it later.

Step 3 – Adding extension files to your project

The source tree of the extension contains two directories: java and cpp. The files correspond to the files needed for your Eclipse project and the files needed for your NDK build.

First, you have to copy the contents of the java folder to the src folder of your Eclipse project. The only modification you need to make to your Java Android project is to instantiate an AdMobHelper object in your activity’s onCreate method. After that your activity’s class should look something like the following example:

public class MyGameActivity extends Cocos2dxActivity {

    protected AdMobHelper mAdMobHelper;

    protected void onCreate(Bundle savedInstanceState){
        super.onCreate(savedInstanceState);

        mAdMobHelper = new AdMobHelper(this);
    }

    static {
        System.loadLibrary("myNativeGameLib");
    }
}

That’s all for the Java part. Now you have to copy the contents of the cpp folder to your C++ source tree (e.g. directly under your Classes folder). If you are using Visual Studio to test your application under Windows, make sure you add the files also to your Visual Studio project.

In order for the new C++ files to get compiled into your native library, you have to add the source file to the list of files to compile in your NDK makefile which you can find in your Eclipse project at jni/Android.mk. After adding the file to the list, your makefile should look something like this:

...
LOCAL_SRC_FILES := hellocpp/main.cpp \
                   ../../Classes/AppDelegate.cpp \
                   ../../Classes/CCAdView.cpp \
...

That’s all about the generic integration steps of the extension. Now it’s time to present how you use it.

Basic usage

The extension adds a new node type to the cocos2d namespace called CCAdView. In order to present your ad, you just have to create an instance of this node and add it to your cocos2d-x layer that you want to present it in. In order to do so, you need to remember two important piece of information when you create your ad unit in the AdMob web interface: the ad type you’ve created and the identified of the ad unit. After having this information, you just have to add the following code to the init method of your layer:

CCAdView* adView = CCAdView::create(AD_SIZE, AD_UNIT_ID);
this->addChild(adView, AD_VIEW_Z_ORDER);
adView->loadAd();

Here AD_UNIT_ID is the string identifier of the ad unit, while AD_SIZE specifies the type (or size) of the ad unit you’ve created and can be one of the constants of the following enum (equivalent with the corresponding Java constants of the AdSize class):

typedef enum _CCAdSize
{
    kCCAdSizeSmartBanner,
    kCCAdSizeBanner,
    kCCAdSizeMediumRectangle,
    kCCAdSizeFullBanner,
    kCCAdSizeLeaderboard,
    kCCAdSizeWideSkyscraper
} CCAdSize;

Remember, these information come from the AdMob web interface. For example, if you created a simple banner then you should use the kCCAdSizeBanner constant, and the ad unit id will be the 15 character long string identifier the AdMob web interface allocated for it.

The value of AD_VIEW_Z_ORDER is actually indifferent on Android as there the actual visual item showing the ad is an Android View, not really a cocos2dx node, thus no matter what Z order you specify the ad will always appear on top of any cocos2d-x rendered primitive. However, the extension comes also with a stub Windows implementation that will show you a white outlined black rectangle as a placeholder of where your ad will actually show up when running your application on an Android device, and as that placeholder is shown as a regular cocos2d-x node on Windows, it is recommended to specify a Z order higher than any other displayed node so that the visual appearance on Windows will match the one you’ll see on the actual device.

You probably also noted that we call the loadAd method of the CCAdView node which is actually equivalent with the Java interface’s loadAd method and it tells the AdMob SDK to load a new ad to the ad unit. You can call this method anytime you want to make sure that a new ad is presented (however, the SDK doesn’t guarantees that a new ad will be shown each time you call this method in order to avoid potential abuse).

Ad placement and visibility

By default, your ad will be shown in the top-left corner of your application. However, you can change its location by specifying its horizontal and vertical alignment with the setAlignment method. For example, in order to place your ad horizontally centered at the bottom of the screen you have to add the following line to your code:

adView->setAlignment(kCCHorizontalAlignmentCenter, kCCVerticalAlignmentBottom);

Currently there is no way to set an explicit position for the ad, but I think for most use cases setting the screen alignment should be satisfactory (at least it was for me).
Besides that, you might also want to hide your ad unit from time to time and then re-show it again later. You can use the setVisible method of the CCAdView node just like you do it with any other cocos2d-x node. In general, it is a good practice to just hide and then show you ad again, instead of deleting and re-creating every single time your whole CCAdView node. Also, you probably want to load a new ad each time you show your ad unit again. These can be easily implemented as follows:

// show new ad
adView->setVisible(true);
adView->loadAd();
...
// hide ad
adView->setVisible(false);

Location based ads

The AdMob SDK also allows you to present targeted ads to the users of your application based on their location. This allows you to show more relevant ads to the user which can both result in better user experience and higher ad revenue. The location used for determining what ads to show can be either based on the user’s coarse-grain location (simple mobile network provided data) or fine-grain location (GPS provided location). In general, the former is perfectly acceptable and is probably less intrusive from the user’s point of view, but for the sake of completeness, the extension supports both types.

By default, the user’s location is not used by AdMob or by this extension. You can check that by using the getUsedLocation method that will return the value kCCLocationNone by default.

If you want to use the user’s coarse-grain location to increase the relevance of ads you have to add the following line to your code:

adView->useLocation(kCCLocationCoarse);

The effect of the useLocation method will take effect the next time you call loadAd.

However, in order to use the coarse-grain location in your application, you also have to request permission to do so in your Android manifest by adding the following line under the manifest element of your XML:

<uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION"/>

In a similar fashion, if you want to use the user’s fine-grain location, add the following line to your C++ code:

adView->useLocation(kCCLocationFine);

While in your Android manifest you have to request the ACCESS_FINE_LOCATION permission instead:

<uses-permission android:name="android.permission.ACCESS_FINE_LOCATION"/>

Note that, however, both of these permissions are optional, and you only need at most one of them, based on what policy you use for location based ads.

Summary

You can see how easy AdMob integration into cocos2d-x projects could be with the extension and the functionality provided is probably enough for most users, at least it was for me. However, I’m pretty new to writing Android JNI code so people who are more experienced in the field would probably find my implementation kind of hacky. That’s perfectly fine, the extension is open source and I’d really like to see people improving it, so feel free to contact me if you have any improvement ideas or patches. The source code is available as a Google Code project here: https://code.google.com/p/cocos2d-x-admob/

Also, the current implementation has the following limitations:

• Only supports Android (and Windows, through a stub that is meant only for development purposes). It would be great if somebody could provide ports to other platforms.
• Only a single CCAdView node is going to work. That means if you create an additional node, that will practically overwrite the state of the previous one. Though I don’t consider this a practical limitation as AdMob doesn’t allow you to display more than one ad at a time anyways.
• Only cocos2d-x 2.x.x is supported, but if you’re still using cocos2d-x 1.x.x I would anyways recommend you to upgrade.

So while there is a lot of room for improvements, I hope that the extension will help people not to have to go through the same problems that I had and can concentrate more on their actual application.

The War Is Far From Over

I’ve chosen the title based on the popular article that tries to prove that OpenGL lost the war against Direct3D. To be honest, I didn’t really like the article at all. First, because it compared OpenGL 3 which targeted Shader Model 4.0 hardware and DirectX 11 which targeted Shader Model 5.0 hardware. Besides that, as we will see, the war is really far from over… This article aims to list the most important features introduced by OpenGL 3.x, OpenGL 4.x, Direct3D 10, Direct3D 11 and we will also talk about the promised features of the upcoming Direct3D 11.1 to be fair with DirectX

After I wrote my article about the latest features introduced in OpenGL someone asked me whether I can write an article about the comparison of the hardware features exposed by OpenGL and Direct3D. Instead of a long explanation, I decided to simply create a table of the features introduced by the APIs. Please note that the list focuses on hardware features and does not discuss API feature differences between the two APIs. The list may be far from complete and I’m happy to get feedback about what is missing from the table so that I can extend it. Also there are features for which I did not find whether an equivalent exists in D3D and are marked with a question mark. If anybody can point me to the answer, I would be happy, but I did not find a specification of the HLSL versions.

[1] There is no support for these counters in OpenGL, however they can be implemented with the help of shader atomic counters.
[2] There is no support in Direct3D to use the dedicated atomic counter hardware (supported currently only by AMD GPUs) only by using an append/consume buffer. Though, as atomic counters are the part of UAVs and arbitrary number of UAVs can be attached to a single resource, the same functionality is supported indirectly.
[3] There is read/write buffer and texture support in Direct3D 11, however it is available only in the fragment (pixel) shader. Direct3D 11.1 plans to remove this restriction.
[4] There is no support for texture format casting in OpenGL, conversion, however, can be done by doing a copy preferably using pixel buffer objects.
[5] There is no support for automatic storage of atomic counter values in buffers in Direct3D, however, their value can be manually copied to arbitrary resources.

As a conclusion, I would like to say just one thing: even though there are some features that are not supported by either OpenGL or Direct3D, we really can say that the two APIs are on par with the number of hardware features they expose.

(Sorry in advance for any mistakes, it took quite some time to create this table and I may became too tired at the end)

New features in OpenGL 4.2

OpenGL 4.2 finally filled the holes in the capability matrix of Shader Model 5.0 hardware with some long waited extensions from which some of the functionalities were actually already accessible through cross-vendor and vendor specific extensions. Also, the new version of the specification brings some important API improvement extensions and GLSL constructs that continue the transition to a more easy to use state and shader management.

ARB_texture_compression_bptc

This extension adds the new block compression texture formats called BC7 and BC6H in Direct3D terminology. The extension is actually available for quite some time, since the release of OpenGL 4.0 but now it became core. The formats provide high quality block compression for fixed point RGBA and sRGB textures as well as two floating point texture compression formats for signed and unsigned data.

Traditional block compression methods (as S3TC or RGTC) use the gradients in a block of pixels which works fine for smooth images but does provide poor results in case of sharp edges. BPTC solves the issue by dividing blocks into multiple partitions which are compressed using independent gradients thus providing better overall quality.

When comparing compression efficiency, BPTC has a compression ratio of 3:1 compared to 6:1, 4:1 and 2:1 that are the compression ratios of the S3TC DXT1, S3TC DXT5 and RGTC formats respectively.

ARB_compressed_texture_pixel_storage

This is an interesting extension that solves a problem that I didn’t even know is such a big issue. The extension is designed primarily to support compressed image formats with fixed-size blocks as that of BPTC as an example. The application can use this extension to configure pixel store parameters so that subtexture operations can provide consistent results in all cases.

ARB_texture_storage

This is again an interesting extension that provides API improvement over how texture storage is allocated in classic OpenGL. As we all know, OpenGL was always too ad hoc on resource management, from the point of view of when actual resources are allocated for a particular API primitive. This is especially a problem in case of textures where we potentially talk about large amount of data. In classic OpenGL the driver could not know from the beginning for example whether the application will need mipmaps for the texture or how many levels are required. This could easily result in bad allocation patterns and/or large reallocations. This extension introduces the concept of immutable texture images where all the levels are allocated up-front for a texture object.

ARB_transform_feedback_instanced

This extension extends the so called “AutoDraw” feature by providing instanced “AutoDraw”. This means that geometry captured using transform feedback can be rendered multiple time using geometry instancing. This is actually a feature that even D3D11 does not provide and being such, I didn’t even think that hardware supports it, even though I think the list usage patterns of the extensions is most probably pretty narrow.

ARB_base_instance

This extension is actually the feature I called ARB_instanced_arrays2 in my suggestion list. The extension provides three new draw commands, one is kind of illy named as DrawElementsInstancedBaseVertexBaseInstance, even though this command can be called the “basic” indexed draw commands that specifies all parameters. Also, the parameter list of the indirect indexed draw command is extended with the base instance parameter. Fortunately, however, the ARB chosen to add new commands rather than a SetBaseInstance-style state specifier command to introduce the new concept. Funnily this feature was missing for a long time as, as far as I know, it is supported by all GPUs capable of doing instanced drawing, and is available in D3D as well.

ARB_shader_image_load_store

This is where things get start really interesting. This new extension is the ARBified version of the extension EXT_shader_image_load_store which fortunately didn’t make it into core in its current form.

The extension provides GLSL built-in functions allowing shaders to load from, store to, and perform atomic read-modify-write operations to a single level of a texture called an image from any shader stage. Also, the extension indirectly enables the same set of operations for buffer objects by using buffer textures. This enables developers to implement more sophisticated algorithms using shaders that require more complex data structures than just plain arrays.

This, together with atomic counters that we will talk about later, enables the possibility to implement append/consume buffers and rendering techniques like AMD’s Order-Independent Transparency (OIT) algorithm as presented at GDC10.

As the introduction of the new write operations to fragment shaders besides the traditional framebuffer writes makes the execution of the shader have side effects and thus sensitive to whether early-Z is used or not by the hardware, so the extension also provides a mechanism to force or disable early-Z in the fragment shader.

A similar issue is in case of vertex shaders as the post-transform cache may be no longer valid in case of certain usage patterns of load/store images so, based on how smart the shader compiler is, the post-transform cache could be easily disabled in case a vertex shader uses load/store images resulting in downgraded performance, so care must be taken when using read/write images in vertex shaders as OpenGL does not have any mechanism to help these issues (but I actually have a proposal that I’ll talk about in a future article).

The API of this extension is greatly improved compared to the EXT version, especially when dealing with various texture image formats. The extension also provides a future-proof DSA-style API. Further, the ARB version of the extension supports loads from any texture format and corrected some specification bugs of the EXT version.

From hardware implementation point of view, it must be noted that in case a shader contains atomic operations applied to a particular read/write image the driver uses a different hardware path, as required by atomic read-modify-writes so that care must be taken to use atomic operations only when necessary. Also note that this decision is made statically at compile time by the driver so even a single atomic operation in an unlikely taken branch will result it degraded performance. This is another reason why to use atomic counters to implement append/consume buffers instead of using read/write image atomics.

ARB_shader_atomic_counters

This the other long waited feature that I also suggested and was still missing from OpenGL but was available in D3D11. The specification was actually ongoing for a long time now (about a year) and it even appeared for a while in AMD’s OpenGL drivers sometimes as EXT, sometimes as ARB extension. The extension provides API to access a number of hardware atomic counters that provide efficient counter operations on a GPU global scale. Atomic counters come handy in many cases like append/consume buffers or indirect draw buffer construction.

The extension provides access to these atomic counters from GLSL and also makes it possible to back them up with buffer objects so after OpenGL draw calls the value of the counters is preserved in these buffers for later use.

The OpenGL implementation is superior compared to D3D’s as it provides access to atomic counters from all shader stages, with caveats of course as, it was mentioned in the previous section, the side effects made possible with read/write images and atomic counters require special care in case of fragment and vertex shaders as they may result in invalid rendering and/or lower performance.

On hardware vendor implementations, it must be noted that atomic counters are much, much more faster than read/write image atomics, at least on AMD hardware which has dedicated hardware for atomic counters. On NVIDIA hardware, though, it seems that there is no different hardware path for atomic counters as their performance is roughly the same as in case of read/write image atomics.

The dedicated hardware implementation of atomic counters, however, comes with a trade-off as the number of atomic counters is severely limited on AMD hardware, but one can still use read/write image atomics if ran out of atomic counters.

ARB_conservative_depth

This is another extension I’ve suggested and that fills another functionality hole compared to D3D11. The extension is actually an ARBified version of AMD_conservative_depth that extends the application developer’s control over eary depth and stencil tests. ARB_shader_image_load_store  already provides a way to force or disable eary-Z and this extension provides further modes that provide a hint to the driver about how depth is modified in a fragment shader that outputs depth. This passes enough information to the GL implementation to activate some early depth test optimizations safely while still preserving the ability to account the final depth value in the depth test.

The extension exposes the new capability in the form of fragment shader input layout qualifiers called “depth_any”, “depth_greater”, “depth_less” and “depth_unchanged”. The interesting ones are the one that assume a greater or less depth value as output and provide the ability to early reject groups of fragments using Hi-Z and early-Z even when depth is modified. This technique can greatly improve the rendering performance of volumetric particles, decals and billboards.

As far as I can tell, though, the extension provides performance benefits only the AMD hardware currently as NVIDIA hardware does not have such functionality thus using the extension would still force NVIDIA GPUs to disable early-Z in case the fragment shader outputs a depth value, but future hardware may change this.

ARB_shading_language_420pack

This is a strangely named extension that provides a lot of improvements to GLSL. These are mostly API improvements only, but have a great value when looking at source code maintainability and resource management.

I think the most useful addition of the extension is the “binding” layout qualifier that I referred to as ARB_explicit_sampler_location and ARB_explicit_uniform_block_index in my suggestion list. This enables shader writers to explicitly bind a uniform block binding index to a uniform block as well as explicitly bind sampler, texture and image binding points to a sampler or image variable.

Besides that, the extension adds other minor improvements, like implicit conversion of return values of functions, UTF-8 character set support, C-style initializer list support and scalar swizzle operators.

ARB_internalformat_query

This is another kind of strangely named extension that was meant to provide the possibility to query information about the internal format of textures, however, it actually failed it as it provides only the ability to query the maximum number of samples available for different texture formats.

The extension was ambitious as it planned to provide internal format information like the ability to query the actual internal format used, whether the format is renderable, accessible in a particular shader stage, whether it can be used as read/write image, and even to provide performance hint about using a particular texture internal format. Unfortunately all these were left for a future extension.

ARB_map_buffer_alignment

This is the last new extension introduced in OpenGL 4.2 that trivially adds the requirement to the pointer returned by buffer mapping commands that they provide a minimum of 64 byte alignment to support processing of the data directly with special CPU instructions like SSE or AVX. This can provide further performance increase when client is modifying buffer data.

Conclusion

OpenGL 4.2 again proven that OpenGL is not dead, but in fact plans to be again the ultimate choice of 3D API by pushing the exposed hardware capabilities over the line set by D3D11. When thinking about the list of expected extensions I presented in my earlier article, Suggestions for OpenGL 4.2 and beyond we can see that OpenGL 4.2 fulfilled all my expectations and even my wish list was partly fulfilled, but here’s the list for a better overview:

My expectations for OpenGL 4.2:

GL_EXT_shader_image_load_store
- added in the form of GL_ARB_shader_image_load_store

GL_ARB_shader_atomic_counters
- added as is

GL_ARB_instanced_arrays2
- added in the form of GL_ARB_base_instance

GL_ARB_explicit_sampler_location
- added in the form of GL_ARB_shading_language_420pack

GL_ARB_explicit_uniform_block_index
- added in the form of GL_ARB_shading_language_420pack

My personal wish-list for OpenGL 4.2:

GL_ARB_draw_indirect2
- still missing, though partly available though GL_AMD_multi_draw_indirect

GL_ARB_direct_state_access
- still missing, however, there is hope that it will be included in the next release where the ARB plans to rewrite the whole structure of the core specification

GL_NV_texture_barrier
- not in core but it is implicitly subsumed by GL_ARB_shader_image_load_store, they say

GL_AMD_conservative_depth
- added in the form of GL_ARB_conservative_depth, despite lack of NVIDIA support

GL_ARB_texture_gather_lod
- still missing, because of lack of supporting hardware

GL_NV_copy_image
- still missing, even though it could be a good API improvement

GL_EXT_texture_filter_anisotropic
- still missing, as I was informed, because of patent issues

GL_ARB_shader_stencil_export
- still missing, most probably because of lack of NVIDIA hardware support

GL_AMD_depth_clamp_separate
- still missing, most probably because of lack of NVIDIA hardware support

GL_AMD_transform_feedback3_lines_triangles
- still missing, most probably because of lack of NVIDIA hardware support

The Evolution

I started the development somewhen in February, Pocket Soccer being my first Android game and, in general, my first real Android application. Actually, I was experimenting with very simple Android demo applications using the emulator already earlier, however, real development was made possible only after January, when I bought my Galaxy S. After roughly a month, I’ve published the first version of Pocket Soccer on the Android Market.

At that time, Pocket Soccer was a bit far from a polished, competitive game, there was no social network integration, no leaderboard, even the training mode was not available yet and, of course it contained quite some bugs. The reason behind the later was that I hadn’t have enough experience yet with the platform and I was also unable to test it on many devices as I only own a single Android device, so I had to ask from time to time my friends to give it a try (the later being still true as I did not have yet the chance to invest on further devices).

Shortly after, Ben Camenker from Scoreloop contacted me with the offer of integrating the Scoreloop social networking framework into Pocket Soccer. Of course, I was interested and thanks to Ben not so much after the first release I’ve integrated Scoreloop into Pocket Soccer, bringing the still great performing rating system and online leaderboard to the game.

Pocket Soccer continued its evolution with the addition of the training mode, that I actually wanted to include from the beginning, but didn’t have the necessary time to do it in the timeframe I wanted to keep. Meanwhile, the game also became much more mature as I was able to track down most of the issues the players reported.

While there are still some bugs hanging around (many of them deeply lying inside the 3rd party libraries and frameworks the game uses) I think the game reached some sort of completeness, even though I’m still planning to add new features to it (however, I also want to allocate some time for a next game as well).

The Numbers

Now, I would like to talk a little bit about the numbers: download counts, active installs and leaderboard members. The game started with very moderate download counts at the beginning. Actually this is the most stressful period of a mobile game developer as here it will turn out whether the game will be noticed or it will go down the drain in the huge pool of apps available for such a popular platform as Android is.

For Pocket Soccer, the great entry happened at the beginning of May, roughly eight weeks after the first release. Since then, more than 350.000 people downloaded Pocket Soccer and there are even currently over 160.000 active installs and around 10.000 active players each day. This result exceeded all of my expectations and I would like to say thanks to all the poeple who downloaded the game and I hope that, at least most of you, have or had a great time playing it. If you don’t mind, I would also like to share some figures with you:

Pocket Soccer total downloads (blue) and active installs (red) over time.

Here, I would like to take the chance and talk a little bit about the alternative Android market places. This may be interesting mostly to fellow developers who are planning to submit their applications to alternative markets. Well, if you ask me, I would say that it not worth the effort, but don’t believe me, believe the numbers…

Shortly after releasing Pocket Soccer on the Android Market, I gave a try to a few alternative markets and download sites, namely to AndroidPIT, SlideME, AppsLib and the Amazon AppStore, Pocket Soccer appearing on the later from the market’s launch. At the beginning even those few downloads meant a lot for me, but after almost four months, the picture is quite disappointing. Here you can see the number of downloads Pocket Soccer acquired at each market so far:

As you can see, the four alternative markets barely contribute to a 4% of the number of total downloads, even though SlideME and the Amazon AppStore can be really thought of as big players of the alternative market business.

I don’t want to discourage any of the developers to give a try to alternative markets as others may have much more success with them, I just wanted to share my personal experience with them. If anybody has much different figures then I would be really interested in it so please take your time and write a comment about it.

The Achiements

Besides the spectacular number of downloads and active players, Pocket Soccer received a lot of other achievements, including many positive ratings and comments, as well as several great reviews, including some video reviews that I would like to share with you (my personal favorite is the second one as this was the first time I’ve seen the game running on a tablet):

http://www.youtube.com/watch?v=-qqoutmSaKs

http://www.youtube.com/watch?v=wkgei2yxTP8

Another thing that I’m very proud of is that Pocket Soccer is in the top 100 free games in the arcade & action category for months now, peaking around the #50 place which I take as a great compliment being a hobbyist and a rookie in the Android world.

Finally, I would like to mention one particular achievement that I’m really proud of, namely that Pocket Soccer was chosen #12 into the Top 25 Droid Apps for Kids in 2011 at Babble (the promotion badge you’ve seen at the beginning of this article).

The Future

While the story of Pocket Soccer already had quite a lot of things to tell, the story is yet far from over. I’ll continue the development on it, even if only with limited time quota, as there are a lot of requests coming from the players and I will continue to try my best to make them pleased, at least, as much as my time allows.

Here, I would like to take the chance to talk about the most requested features to try to explain why they are not available yet and/or when they can be expected:

• Tournament Mode (or World Cup, Campaign or Career mode if you wish)
• Networked Multiplayer Mode (bluetooth, WiFi and/or online)

Implementing a tournament mode may not seem like a complex, and actually it may or may not be in practice either, however, please consider something: I’m not a full-time game developer, I’m working at a multinational company 40 hours a week and lately I was also occupied with my exams and master thesis because I’ve just acquired my master degree lately. Spare time is really of scarcity in my life, so please forgive me for not making it already. I’m seriously willing to make it, just I have my own limits as well.

About the networked multiplayer mode, well, I have problems with making that. I don’t think I’ll make an online multiplayer mode because, besides that being the biggest effort, I’m afraid the latency, especially in case of mobile internet, would be highly prohibitive, as for such a fast paced game like Pocket Soccer, even a small delay could be a show stopper. So I would go rather in the direction of a bluetooth or WiFi based multiplayer mode.

Finally, I would like to make some further games that I need to spend time with as well, so I hope you understand now why those so asked features didn’t make their way into the game yet. This won’t stay so forever and I’m really willing to implement all the features you request.

Summing up everything, Pocket Soccer brought a great amount of achievements to me and, hopefully, a great amount of fun to the players! And, of course, the story does not end here as I will continue the development…

The proposition

First of all, let’s recap what exactly GL_ARB_draw_indirect brought us:

This extension provides a mechanism for supplying the arguments to a DrawArraysInstanced or DrawElementsInstancedBaseVertex from buffer object memory. This is not particularly useful for applications where the CPU knows the values of the arguments beforehand, but is helpful when the values will be generated on the GPU through any mechanism that can write to a buffer object including image stores, atomic counters, or compute interop. This allows the GPU to consume these arguments without a round-trip to the CPU or the expensive synchronization that would involve. This is similar to the DrawTransformFeedbackEXT command from EXT_transform_feedback2, but offers much more flexibility in both generating the arguments and in the type of Draws that can be accomplished.

If you know my Nature or Mountains demo you know that I have dug deeply into the domain of GPU based culling algorithms. In case of these algorithms, the GPU consumes the scene data and performs visibility determination over a list of objects and writes out the culled data into a buffer object. The problem is that those algorithms that I’ve implemented in the aforementioned demo applications work only for instanced objects. In order to make it possible for the algorithms to be able to efficiently work with arbitrary object sets we still need a lot of new features (some of them may even require newer GPU generations). The most important ones are discussed in detail in the following sections.

Atomic counters

This feature enables us to use the global atomic counters present on the GPU, which have, at least on the AMD implementation, dedicated hardware to provide efficient chip-wide access to these counters from any shader. This can be expected in the near future in the form of the yet not published GL_ARB_shader_atomic_counter extension. The extension also provides a way to back up the atomic counter values in buffer object memory.

The currently available GPU based culling algorithms, including those presented in my demos, bypass the lack of this feature by using transform feedback to capture the culled data which has implicit atomic counters that are associated with each output stream. However, this has a few drawbacks. First of all, transform feedback is not as efficient if one would use atomic counters together with the random memory read/write mechanism exposed by the GL_EXT_shader_image_load_store extension. This is because of its nature, geometry shaders and thus transform feedback has to preserve the original order of the incoming primitives. This is why the first GPU generation with geometry shader support had so much performance problems as the use of geometry shaders easily became the bottleneck of the rendering. Besides the performance benefits of having our own atomic counters, there are a lot of other reasons, like the ability to implement an append/consume buffer, if I’m allowed to use the D3D terminology.

It may seem that I went a bit off-topic, however, just think about how atomic counters can interact so nicely with indirect drawing. There is the instance count field of the indirect draw commands, what if we bind that address as the back-up buffer memory for the atomic counter? Yes, we can save that costly asynchronous query to get the number of visible objects that we did otherwise in case of applying an ICR or Hi-Z map based occlusion culling. You may say that you can achieve the same thing with atomic read/writes as provided by the GL_EXT_shader_image_load_store. Well, that’s true, unless the additional performance hit by doing atomic memory writes is acceptable (atomic counters are much, much faster, however, it is true that in case of a GPU based culling algorithm, those few writes shouldn’t be the bottleneck). But now let us think more deeply into the problem. If we can use atomic read/writes to count the instances, as it is present in the indirect draw command in the buffer object, then what if we count the number of draw commands written into the indirect draw buffer using atomic counters? And here we are, we have the first building block of a GPU based culling algorithm that can handle arbitrary data sets.

Multi-Draw-Indirect phase 1

Now let’s say we somehow managed to generate an indirect draw buffer object with the list of the instanced draw command arguments necessary to render the visible objects, no matter whether we used the OpenGL toolset as in my demos or we used some compute API like OpenCL. Now somehow we have to initiate the drawing. We can do this by issuing several DrawArraysIndirect or DrawElementsIndirect command based on how many instanced draw command arguments we’ve generated.

But what if we could do this with a single command? This is where GL_AMD_multi_draw_indirect comes into picture and that’s what AMD implemented for us. We can actually do this by using one of the MultiDraw*Indirect commands introduced by the extension.

The best thing in it is that in case of lack of hardware support for it, the driver can still implement it by simply making a loop that calls the appropriate Draw*Indirect commands so every hardware that supports GL_ARB_draw_indirect can support GL_AMD_multi_draw_indirect, and in case the hardware actually supports the functionality, then we can get a slight performance increase for free.

Multi-Draw-Indirect phase 2

While the new extension adds quite some flexibility to the existing indirect drawing mechanism, it still lacks an important feature to become the Holy Grail of GPU based culling and scene management algorithms. We still have to perform an asynchronous query or otherwise determine the number of records written into the indirect draw buffer.

Of course, we can alleviate the problem by always initializing the indirect draw buffer with zero values (so that if one would issue an indirect draw command using any of the data in the buffer no actual rendering would take place) and then simply using a MultiDraw*Indirect command passing a primcount argument that is equal to the theoretical maximum of generated records. However, this might result in a performance decrease, especially if this theoretical maximum value is much bigger than the actual draw commands present in the buffer.

In order to circumvent this problem, we need some mechanism that allows us to also source the primcount argument of the MultiDraw*Indirect commands from buffer object memory. While such functionality is not exposed yet by any of the major graphics APIs (and may not be supported by current hardware) this could be the next major step towards a fully self-feeding renderer that handles graphics related data on a much higher level beyond triangles and pixels.

Conclusion

While the indirect drawing mechanism introduced with OpenGL 4.0 is just a very little part of the feature set introduced by Shader Model 5.0 GPUs, it has still a lot of room for improvement and evolution ahead. AMD made the first step with GL_AMD_multi_draw_indirect and I really hope that indirect drawing and other GPU self-feed mechanisms will gain more developer attention in the near future.

Finally, I would like to thank to Graham Sellers, the creator of the extension, Pierre Bourdier for his support on promoting the new functionality and all the engineers at AMD who have contributed to the specification and the implementation work behind it. I’m really glad to see that they take the word of the developers in which direction they improve their OpenGL support.

The game itself is a reinterpretation of a classic board game called button football that is very popular in my home country. The key difference is that the game does not contain the many rules like the original one to provide a smoother and more fast-paced game-play. Each player has three buttons that they control by grabbing and throwing them in the desired direction. If one manages to push the soccer ball into the opposite player’s goal then he or she gets one point. The first one to reach ten points wins the match.

The game is turn based so each player has five seconds to move with one of his/her buttons. While, in my opinion, the game is more fun in two-player mode when two buddies can play against each other on the same device, the game also features a pretty smart AI with three difficulty levels. But that’s enough talk, maybe some screenshots say more:

Starting lineup in a match between Spain and Portugal.

Besides the possibility to choose between more than sixty countries to play with, the game has also other changeable assets like different soccer fields and balls. These also come with different physical properties that slightly change the game-play. While some of these assets come out-of-the-box, some others are only accessible if you unlock them. You can do so by playing and/or winning a number of matches in the various game modes. The prerequisites of each asset can be checked in the appropriate menu and you can also check your current accomplishments by tapping the statistics button in the main menu.

Another match between Peru and Uruguay. The player with Uruguay is about to move.

The game should work well on most Android devices. It requires only API level 4 (Android 1.6). I’ve mainly tested it on my Samsung Galaxy S, which of course runs it smoothly, but I tested it also on other devices like the Motorola Droid, ZTE Blade (San Fransisco) and Samsung Galaxy Spica. The game worked well on the Droid and especially smooth on the Blade, which surprised me a little bit for such a cheap phone. In case of the Spica, it already felt that the phone was not made for gaming, however, at the end I managed to optimize the game enough so that it provides a good user experience on that phone as well.

The main menu. You can scroll left and right to access the additional menu items and you can check your statistics anytime by tapping its icon in the bottom-right corner.

I tried to make the game look like the least possible like “programmer art” and I home I managed to do so. In order to have a fast time-to-market with my first game, I’ve chosen to use a game engine framework first, rather than writing my own. Having a lack of native game engines for Android, I settled down with AndEngine as it looked to have a fast learning curve and actually it has (other option was libgdx). While I’m not a great fan of pre-cooked solutions, AndEngine worked out pretty well with its native Box2D accessible over JNI, however, I also had some bad experiences. I will write another post about my development experiences with Android and AndEngine.

Summary

To sum it up, I managed to publish my first game and I hope you’ll like it. The game is ad supported, so you can download it for FREE from the android market:

Again, there was quite a pause in the flow of articles. This is because I’m currently working on a project that consumes most of my spare time and unfortunately takes away time from the update of the blog as well. I would like to take the opportunity and say a few words about this project…

During the last month I worked on the development of an Android game that I plan to release in the near future. It is my first serious try in developing games for handhelds and I hope people will like it. Unfortunately, I’m also one of those fanatic developers who were always interested in game development and always targeted the high end PC gaming industry.

Like many fellow developers, despite my knowledge and enthusiasm, I also struggled with lack of resources, especially considering non-programming related stuff like artwork, which resulted in several partly finished but abandoned projects. This is the reason why I finally settled down in first concentrating on less-demanding yet interesting portions of the industry: mobile game development.

While making games for mobile platforms is usually less ground-breaking and revolutionary, at least I can finally publish something to the end-users in the hope that the gathered experience and notoriety may enable me to target also PC gaming in the future. But this is not all about it… In fact, despite the difficulty of targeting an embedded platform, developing games for phones is great fun and I can advise it to every developer to at least try it out.

Stay tuned for news about the project, especially if you own an Android phone!

Frei-Chen edge detector

In this article, I would like to present you an edge detection algorithm that shares similar performance characteristics like the well-known Sobel operator but provides slightly better edge detection and can be seamlessly extended with little to no performance overhead to also detect corners alongside with edges. The algorithm works on a 3×3 texel footprint similarly like the Sobel filter but applies a total of nine convolution masks over the image that can be used for either edge or corner detection. The article presents the mathematical background that is needed to implement the edge detector and provides a reference implementation written in C/C++ using OpenGL that showcases both the Frei-Chen and the Sobel edge detection filter applied to the same image.

I met with the algorithm during my computer graphics studies when one of my homeworks was to implement the Frei-Chen edge detector. As I already mentioned it in an earlier post, I am willing to provide source code for more basic graphics algorithms after seeing the success of my former post about the Gaussian blur filter. This one is a very similarly basic article, taking in consideration it shows only how to apply a particular convolution filter based algorithm on a still image, while the possibilities this edge detection algorithm brings is a more complex topic that is out of the scope of this article.

As the provided reference implementation also showcases applying the Sobel operator on an image, I would like to present that first and then continue with the presentation of the Frei-Chen masking set. Those who are already well familiar with edge detection and the Sobel operator can skip the following two sections.

Edge detection

Before getting deep into how to implement edge detectors, let’s first talk about what is an edge detector and why we need it.

In general, edge detection is one of the most fundamental image processing tools, particularly used in the areas of feature detection and feature extraction. The aim of the technique is to identify points of a digital image at which the intensity changes sharply. The reason of these intensity changes can be either discontinuities in depth, surface orientation, lighting condition changes and many other factors. In the ideal case, the result of applying an edge detector to an image leads us to a set of connected lines or curves that indicate the boundaries of objects.

Not going that far, what an edge detector gives us from the very beginning is a gray-scale image where each pixel intensity tries to approximate the likelihood of whether that pixel belongs to an object boundary. How well a particular algorithm can detect such pixels depends on many factors and usually it is better to try multiple edge detectors in order to choose one that fits most for the particular use case.

After we got this gray-scale image we usually have to define a threshold value that will be used as an acceptance criteria for edge pixels. If the intensity value previously calculated is above this threshold then we accept the pixel as an edge otherwise we don’t. This part is the so called binarization stage. Additionally, subsequent image processing algorithms can be used to further interpret the edge image.

In computer graphics, edge detection is usually used to implement various image decoration algorithms. Maybe the most popular applications of edge detectors nowadays are non-photorealistic rendering (NPR) and screen-space anti-aliasing techniques.

Sobel filter

The Sobel edge detection filter works on a 3×3 texel footprint and applies two convolution masks to the image that are intended to detect horizontal and vertical gradients of the image. The filter weights can be seen in on the figure below:

These masks are applied to the intensities gathered from the 3×3 footprint of the image and then are accumulated to produce the final gradient value in the following way:

The actual algorithm can be seen in the accompanying demo that provides a GLSL based implementation. The algorithm is defined to work on one channel image, however it can be easily extended to be applied either separately on a usual three-channel RGB image or by first calculating a gray-scale value based on the color component values. The former is more computationally intensive but usually provides better results by defining the threshold criteria in a way that a pixel is accepted as boundary point if the gradient value is larger than the threshold for either of the color channels. The reference implementation, however is based on the later approach for the sake of simplicity so for each pixel first an intensity value is calculated simply by taking the length of the vector comprised of the RGB components.

Frei-Chen filter

The Frei-Chen edge detector also works on a 3×3 texel footprint but applies a total of nine convolution masks to the image. Frei-Chen masks are unique masks, which contain all of the basis vectors. This implies that a 3×3 image area is represented with the weighted sum of nine Frei-Chen masks that can be seen below:

The first four Frei-Chen masks above are used for edges, the next four are used for lines and the last mask is used to compute averages. For edge detection, appropriate masks are chosen and the image is projected onto it. The projection equation is given below:

When we are using the Frei-Chen masks for edge detection we are searching for the cosine defined above and we use the first four masks as the elements of importance so the first sum above goes from one to four.

The application of a threshold and applying the filter to multi-channel images works exactly the same way like in case of the Sobel filter. Similarly, the reference implementation applies the filter on the image as it would be a single-channel image by first calculating the intensity value for each texel in the same fashion like with the previously presented filter.

Comparison

Based on my experience, the Frei-Chen edge detector looks better than the Sobel filter as it is less sensitive to noise and is able to detect edges that have small gradients and thus are not found by the basic Sobel filter. For a comparison, you can check the figure below:

Comparison of edge detectors: original image (left), Sobel filter (middle), Frei-Chen filter (right).

The reason why the Frei-Chen edge detector seems to work better is because its construction includes a normalization factor as well as other factors that are meant to exclude all other features except edges. A normalization factor can be also added to the Sobel filter by having a third mask that is equivalent with the ninth Frei-Chen mask and is used to normalize the gradients. This could help in reducing the number of undetected edges and the amount of noise that arises from the fact that the Sobel filter calculates absolute gradients rather than relative ones.

From performance point of view, the Frei-Chen edge detector is much more heavyweight as it uses nine masks instead of two, however, in practice, the performance difference between the two is much less taking in consideration that both use the same sized texel footprint and the computational performance of today’s GPUs is usually much higher than their texture fetching performance.

Conclusion

We managed to present an alternative algorithm for the Sobel filter in the form of the Frei-Chen edge detector that, even though having little impact on the performance compared to the Sobel operator, provides better edge detection quality. Having little to no difference in the way how the input data has to be organized and how the result is output, the Frei-Chen edge detector can be easily used as a drop-in replacement for implementations that used the Sobel filter before.

Source code and Win32 binary can be acquired in the downloads section.

I would like to encourage those who read this article to add the Frei-Chen edge detector into their software for making a comparison about whether it yields to better results than the Sobel filter for applications that rely on the output of the edge detection filter. I would be interested how the filter works in real-life computer graphics scenarios.

Thanks in advance and hope you enjoyed the article!

Happy Birthday !!

After a quite long, about two months intermission, let me celebrate for a moment the birthday of the blog. It was roughly a year ago that I started the RasterGrid Blogosphere with the wish to have a forum where I can share my experience and ideas with others. Meanwhile, I posted several programming related articles about graphics, multiprocessing and published a few demo applications. However, this is not the best part of it… The thing that I’m most proud of is that the blog now has many returning visitors whom I was somehow able to help and that I’ve received a lot of feedback and ideas from you. I’m especially pleased because the list of people who contacted me via the comments section or via e-mail contains several great personalities of the industry with who I wouldn’t be able to meet if I would not start this project. Thank you all for this!

I’m really happy to announce that I will continue to bring you topics in the second season of the RasterGrid Blogosphere.

Most probably you’ve noticed that I haven’t written for a while. Sorry for that, sometimes it is really difficult to allocate time to sit down and write, especially because I always tried to focus on quality, what I hope I more or less did. I also wanted to publish much more demonstration applications but making these even takes more time than just writing. Actually many topics have been dropped from the content of the previous year just because they would have been valuable only if they would be accompanied by source code as well.

Now let’s not talk anymore about the past, let’s concentrate what will come in 2011. While I don’t want to make any more promises about average number of posts per month as it didn’t work in the past either, I still would like to indicate about what I will write.

As you may observed, even though the blog was started as a general programming related blog, at the end I could not deny myself and it ended up in more like a graphics programming blog. Sorry for that if you expected more general programming articles as I will most probably continue in this fashion in the new year having the following key topics:

• More advanced graphics programming articles with focus on OpenGL 3+ and possibly on OpenGL ES and DX11.
• More graphics demos showcasing advanced rendering techniques, focusing more on OpenGL 4+ functionality.
• New! More tutorial-style articles about less-complicated graphics topics after seeing the popularity of my article written about Gaussian blur. Those who read my curriculum vitae know that besides working at Nokia Siemens Networks, I’m doing my master degree about computer graphics and image processing. Meanwhile, I had to make a few programming exercises related to computer geometry and signal processing and as they are quite basic ones that showcase a particular topic I would like to share these codes with you as well as part of the tutorial sessions.
• New! Another new thing that will come with the new year that is based on your, the reader’s request is the new Tools category that will contain articles about various development tools, especially focusing on applications and libraries that ease and assist OpenGL development.
• As time will allow and as inspiration will be available, I will write also about general programming related articles. The following domains are on my wish-list: OO patterns “why” and “when”s, more unit testing, asynchronous pipelines and other multiprocessing models, etc.

Of course, the list is not something written in stone, every idea is welcomed!

Thanks again for your attendance and interest so far!

Since the original OpenGL Longs Peak announcement, graphics developers were really excited to get their hands on the completely revised OpenGL 3 specification. Still, due to severe backward compatibility and portability issues the original plan seemed to be failed and developers expressed their great sense of disappointment about the ARB’s decision to choose rather a more evolutionary move away from the legacy API instead of the radical rewrite, the Khronos Group has proved that the decision was not necessarily bad for OpenGL and in fact we got now a pretty powerful API, even though the coexistence of the legacy and the new design greatly increased the complexity of the specification.

What we have now is an API that can really compete with DirectX 11 but I strongly believe that this is not the end of the story yet as we still have a lot of things to do in ahead of us. I mean this both from point of view of exposing more hardware capabilities as well as streamlining the API language itself to increase the productivity of the developers who use it. My plan is to target both of these issues in this article, also trying to focus on hardware functionalities that are not even exposed by other graphics APIs yet.

Exposing more hardware capabilities

In this chapter of the article I will talk about some familiar and some not so familiar hardware features and corresponding OpenGL extensions that should be included in the next revision of the specification in order to be able to confidently say that OpenGL is a strict superset of the competing graphics APIs. The extensions listed here are not in any particular priority order, they are just listed in a way that ease the discussion about their functionality.

GL_EXT_shader_image_load_store

This extension provides GLSL built-in functions allowing shaders to load from, store to, and perform atomic read-modify-write operations to a single level of a texture from any shader stage. Also, the extension also indirectly enables the same operations for buffer objects by using texture buffers. This enables developers to implement more sophisticated algorithms using shaders that require more complex data structures than just plain arrays.

An example use case can be the implementation of Order-Independent Transparency (OIT) using fragment linked lists as presented by AMD at GDC10. Of course, there are a lot of other techniques that could benefit from hardware accelerated random access images (called UAV textures/buffers in DX11 terminology) including algorithms related to global illumination, ray tracing, and my personal favorite: scene management.

As the introduction of new write operations to fragment shaders besides the traditional framebuffer writes makes the execution of the shaders sensitive to whether early-Z is used or not by the hardware, the extension also introduces a new fragment shader input layout qualifier called “early_fragment_tests” to force OpenGL to use early depth and stencil test. Otherwise the specification language is valid stating that the depth and stencil tests are performed after fragment shader execution.

Finally, the extension enables some form of control over the order of image loads, stores, and atomics relative to other pipeline operations accessing the same memory region both using the OpenGL API and from within shaders.

The API itself provides a DSA-style binding mechanism that enables binding to so called “image units” that are separate from that of texture image units. In the same style, the specification language and GLSL refers to the introduced read-write textures with the term “image”.

In my opinion this is one of the most important extensions that should be made core with OpenGL 4.2 and I’m pretty sure this will actually happen.

GL_NV_texture_barrier

This extension relaxes the restrictions of OpenGL on rendering to a currently bound texture and provides a mechanism to avoid read-after-write problems. More precisely, the extension allows rendering to a currently bound texture in the following cases:

• If the reads and writes are from/to disjoint sets of texels (after accounting for texture filtering rules) so it should work unless the drawn areas overlap, or
• If there is only a single read and write of each texel, and the read is in the fragment shader invocation that writes the same texel (e.g. using texelFetch2D).

Some of these situations were already supported implicitly like rendering to a texture level and fetching from another texture level. But the extension goes further and provides an API function to put an explicit barrier between draw calls to ensure proper rendering.

The extension can be used to accomplish a limited form of programmable blending and can eliminate the need of any image or buffer data copy in case we can live with the restrictions mentioned above.

One may ask why we need this extension if we have the GL_EXT_shader_image_load_store extension as this one is just a subset of the functionality provided by that. The answer is simple: performance. While read-write textures can mimic the same functionality they usually use different hardware paths that are slower than regular read-only texture accesses. So it would be a definite benefit to having also this extension in core OpenGL.

GL_ARB_shader_atomic_counters

This extension does not have public specifications yet, however it can be found in the extension lists of the latest Catalyst driver releases sometimes with EXT, sometimes with ARB prefix. The extension itself provides API to access a number of hardware atomic counters that provide efficient counter operations on a GPU global scale.

Atomic counters come handy when one has to read or write individual elements of a buffer or texture. As an example, this extension is needed to be able to efficiently implement the OIT algorithm mentioned earlier as, when constructing the fragment linked list, we need to have unique offsets to the linked list buffer. This unique offset can be, of course, acquired by using atomic read-modify-write operations but those perform much slower than hardware atomic counters.

Besides the mentioned example, atomic counters are useful in many algorithms from many domains, one important use case is to perform feedback operations similar to that provided by transform feedback. Such feedback operations can be used to perform various scene management or culling mechanisms.

The extension provides access to these atomic counters from GLSL and also makes it possible to back them up with buffer objects so after OpenGL draw calls the value of the counters is conserved in these buffers for subsequent use.

GL_AMD_conservative_depth

Early depth test is a common optimization for hardware accelerated graphics that can skip the evaluation of fragment shaders for fragments that end up being discarded because they don’t pass the depth test. The problem is that in case the fragment shader modifies the depth value of the fragment then the early depth test is disabled. One can force early depth test with the functionality introduced by the extension GL_EXT_shader_image_load_store but that can lead to some rendering artifacts as the modified depth value output by the fragment shader is not taken into account.

This extension allows the application to pass enough information to the GL implementation to activate some early depth test optimizations safely while still preserving the ability to account the final depth value in the depth test. In order to solve this, the extension introduces four new fragment shader input layout qualifiers called “depth_unchanged, “depth_any”, “depth_greater” and “depth_less”. The most interesting ones are the latest two that provide the ability to do early-Z and hierarchical-Z tests from one direction to discard some groups of fragments and still allow the fragment shader to safely modify the depth value.

This technique comes very handy in case of rendering volumetric particles, decals or billboards. Without this extension one have to sacrifice the possibility to do early rejection of fragments in order to be able to create the volumetric primitives mentioned.

As far as I know this feature is also present in DirectX 11 so it should be a must for OpenGL 4.x also. As the extension is an AMD one, I don’t know whether NVIDIA GPUs do support anything like this in hardware but even if not, they can simply ignore the new layout qualifiers and do late depth test instead. Of course, it would result in lower performance but if only functionality is concerned it should be just okay.

GL_ARB_instanced_arrays2

OpenGL provides two means to perform geometry instancing via the extensions GL_ARB_draw_instanced and GL_ARB_instanced_arrays. While this (yet non-existent) extension would extend both, it is more relevant in case of the extension mentioned later so I named it accordingly.

The extension should trivially add the possibility to specify a “first instance” parameter for the instanced draw commands. Whether this is accomplished by introducing new variants of the glDrawElement* and glDrawArrays* draw commands or having a separate command for specifying the new parameter is up to the ARB. The extension should also interact with GL_ARB_draw_indirect which already mentions the lack of the parameter in GL and reserved already a field in the indirect draw command structure for specifying the “first instance” parameter.

This extension itself would be much more a bug fix rather than a completely new feature as this functionality should have been already exposed at the first time instancing was introduced to OpenGL.

GL_ARB_draw_indirect2

This is one of the extensions I would be the most happy to see in the next release of the OpenGL specification. It would be a functional addition to the GL_ARB_draw_indirect extension that currently only allows the execution of a single instanced draw command that sources its parameter from a buffer object.

The new extension would add a new buffer binding point called e.g. GL_DRAW_INDIRECT_PRIMITIVE_COUNT that would specify the source of the “primcount” parameter to the following newly introduced draw commands:

    void MultiDrawArraysIndirect( enum mode, sizei stride,
                                  const void *indirect,
                                  const void *primcount );
    void MultiDrawElementsIndirect( enum mode, enum type, sizei stride,
                                    const void *indirect,
                                    const void *primcount );

This would not just allow for executing multiple indirect draw commands at once, without further CPU action, but also would source the “primcount” parameter from a buffer object thus if the draw commands are generated using transform feedback, read-write buffers or OpenCL (e.g. based on some GPU based scene management algorithm) then the application does not have to use asynchronous queries or other means that may introduce sync points in the rendering to be able to feed the “primcount” parameter.

Some people said that this is quite a futuristic feature to expect and most probably such functionality will be available only on newer generation of GPUs and maybe with OpenGL 5. I was not that pessimistic so I decided to raise my question to the relevant ARB members of NVIDIA and AMD. While I did not receive any answer from NVIDIA, I did received some good news from AMD as they said that this functionality can be implemented for Shader Model 5.0 level hardware.

What this extension would give developers is a way to efficiently implement GPU based scene management where the GPU bakes together all the rendering commands for the current frame using atomic counters and buffer writes, and the CPU just have to issue a few or maybe just a single MultiDraw*Indirect command to render the whole scene. But of course, the feature can increase draw command throughput also in case of CPU based scene management.

So my message to the Khronos Group is please, start working on such an extension as this would not just make developers happy, but you can also strengthen OpenGL’s position in the industry by putting something into the specification that even DirectX 11 cannot do.

GL_AMD_transform_feedback3_lines_triangles

OpenGL 4.0 introduced the extension GL_ARB_transform_feedback3 that further extended the transform feedback capabilities provided by earlier extensions to allow ouput to separate vertex streams. However there is one caveat: separate vertex streams are only supported for point primitives.

This new AMD extension does nothing more than just simply removes that restrictions for separate output streams allowing the same set of primitive types to be used with multiple transform feedback streams as with a single stream as long as the primitive types are the same for all output streams.

Limiting the possible output primitive types for transform feedback into multiple streams should not be a problem unless you want also to rasterize some triangles at the same time you output. Without relaxing this restriction can do this only by issuing two separate draw commands that incurs a performance hit.

I don’t know if the restriction is present in the ARB extension because NVIDIA does not support this in hardware but if this is not the case then I think this extension should be included in the next release of the specification. Otherwise, please NVIDIA include this feature in your next GPU generation.

GL_NV_copy_image

OpenGL 3.1 already introduced a method to provide GPU accelerated copy of buffer data. This NVIDIA extension provides a similar functionality that can be used to execute efficient image data transfer between image objects (i.e. textures and renderbuffers).

While there are already methods to perform image data copies between textures e.g. using the GL_EXT_framebuffer_blit extension promoted to core with OpenGL 3.0 these require expensive framebuffer object operations and they also lack direct support for transferring 3D image data.

This extension simply introduces a single command that allows such image data copies for every type of textures (including cube maps, 3D textures and array textures) without the need to bind the image objects or otherwise configure the rendering.

GL_AMD_depth_clamp_separate

The extension GL_ARB_depth_clamp promoted to core with OpenGL 3.2 introduced the ability to control the clamping of the depth value for both the near and far clip planes. This eliminates artifacts like seeing inside an object happening when the object’s geometry is clipped by the near clip plane.

This new extension provides a mean for the application to enable depth clamp separately for the near and the far clip plane. This increases the flexibility of depth clamping and can save some fill-rate in certain situations.

GL_EXT_texture_filter_anisotropic

I don’t think that I have to talk too much about this extension as it should be familiar to all of you. It simply enables the possibility to use anisotropic filtering on a per-texture basis. I really wonder how this extension didn’t make its way into core as it is supported by hardware since more than a decade.

I know that the extension itself is supported by all relevant graphics driver vendors but really, why we can’t just simply include it in the core specification?

GL_ARB_texture_gather_lod

This is another yet non-existent extension that would extend GL_ARB_texture_gather by adding GLSL built-in functions called textureGatherLod that would allow gathered fetches with explicit LOD. I’m not sure if these functions are missing from the specification because of lack of hardware support or just because the ARB thought they might not be of any use. Anyway, if the hardware supports it then OpenGL should expose it to developers as there are certain situations when one has to use explicit LOD and could benefit from the increased fetching performance enabled by gathered fetches.

GL_ARB_shader_stencil_export

This extension was published at the time the OpenGL 4.1 specification came out and provides the ability for the fragment shader to output the stencil reference value that was otherwise configurable only using API calls. This enables a great level of flexibility to existing and future stencil buffer based algorithms making it possible also to directly write independent values to the stencil buffer on a per-fragment basis.

The predecessor of the extension is GL_AMD_shader_stencil_export and as such it indicates that maybe it is only supported in hardware on AMD GPUs. However, if this is not the case and NVIDIA could support this also then I think it worths to promote this feature also to core OpenGL.

Streamlining the API

After discussing the long list of functional features that would be nice to be included into the next release of OpenGL let’s focus on the API improvement extensions and ideas that are necessary to improve the usability of the API itself. Actually this part could go way longer than I’ll discuss because as we get more and more features to OpenGL, developers struggle with the increased complexity of the API. I’ll try to focus on the most crucial issues.

GL_EXT_direct_state_access

This is the extension what all OpenGL developers are waiting for a long time now. Direct state access eliminates the OpenGL API’s stupid “bind-to-modify” nature.

For a very long time the only vendor supporting the extension was NVIDIA. Fortunately, since Catalyst 10.7 AMD also exposes the extension to developers. Still, I have one problem: this extension is very poorly designed.

The main problem with the extension is that the functions were designed in a way that a naive implementation could be done by simply using “bind-to-modify” under the hood. That’s what resulted in crazy API functions like MultiTexParameter* and friends. Also, enabling DSA for all of the deprecated functionalities would result in an explosion of the API specification and as a consequence it would result in bloated specification language. Finally, I would also like to object somewhat the lack of creativity of the contributors regarding to the awkward naming conventions present in the current DSA extension.

In my opinion the Khronos Group has to address the issue by creating a new ARB version of the DSA extension that focuses strictly on core functionalities, throwing away DSA support for deprecated features (if somebody needs to use deprecated features they can still use the EXT version) and provide a naming convention that fits much better into the current API language.

Anyway, I completely agree with the other developers out there and scream for DSA. I think the Khronos Group has to eliminate the problem of the “bind-to-modify” semantics as soon as possible otherwise, even though the core specification exposes more and more hardware features, developers will not be attracted to use OpenGL.

GL_ARB_explicit_sampler_location

The ARB moved in the right direction when they introduced the GL_ARB_explicit_attrib_location extension by eliminating the need to use dummy API calls to bind vertex attributes and output buffers to shader variables but they should not stop here. One of the most important addition could be adding a similar language syntax to GLSL that would allow us to bind sampler uniforms to texture image units. Obviously, the same goes for read-write images if GL_EXT_shader_image_load_store is included.

GL_ARB_explicit_uniform_block_index

Similar to the previous request, uniform block indices should be as well explicitly specifiable in the shaders themselves. This extension would add exactly such functionality. The implementation is also straightforward: just a simple uniform block layout qualifier has to be added.

Other API clarifications

Besides the major issues the current specification language also has some bugs and unclear parts that should be addressed as well:

• Program pipeline objects are created by binding the object name which is not in align with the rest of the API language.
• No language is about whether program pipeline objects are shared among contexts or not which suggests that they aren’t which is not in align with the fact that program and shader objects are shared.

Most probably there are a lot more issues with the specification language but for now just these came into my mind. Maybe some of you can extend the list with tons of other specification mistakes.

OpenGL 4.2 and beyond

While my feature requests cover most of the needed functionality that should be included in the next revision of the OpenGL specification, there are a lot of other things that could be very useful for developers but are very unlikely to get their way into the specification any soon. I will talk about these features in this section of the article as these raise much more questions than just to be able to simply include it in OpenGL 4.2.

Affinity contexts

We have multi-GPU designs like SLI and CrossFire for a long time now. Fortunately, we have also vendor specific extensions to create affinity contexts that are associated with a single GPU of a multi-GPU configuration. We have WGL_AMD_gpu_association and WGL_NV_gpu_affinity for Windows and GLX_AMD_gpu_association on GLX based platforms. I have just two problems with this:

• First, these are vendor specific extensions.
• Second, NVIDIA exposes its affinity context support only on Windows and just for their professional cards, leaving consumer hardware owners without affinity context support.

I would be pleased to see in the future extensions like WGL_ARB_gpu_affinity_context and GLX_ARB_gpu_affinity_context that will be supported by both NVIDIA and AMD, and that are supported on both professional and consumer hardware.

Command buffers

I would like to see something similar in OpenGL that what we have in OpenCL. Having several separate command buffers for a single OpenGL context can have its performance benefits as some of the implicit sync points that are otherwise present in OpenGL draw commands could be eliminated. Another solution would be to use simply multiple GL contexts but it is much more complicated and context switches are quite heavy-weight operations. This would be something like how framebuffer objects replaced pbuffers.

Also this could go that far as we can encapsulate state manipulation data into command buffers in a similar way how display lists allowed this in many cases just in a more efficient and hardware centric manner.

Immutable state objects

Another thing strongly related to the previous idea would be immutable state objects. If state management data could not be efficiently stored in such a command buffer we could use instead immutable state objects that would be very similar in nature to display lists that are hiding the underlying representation of the commands.

Display lists are deprecated and I don’t think it was a wrong decision. It made the API language complex and you’ve never knew which command compiles into display lists and how. I remember the time I was making an OpenGL app on my GeForce2 and used DrawElements calls inside display lists that referenced buffer object data. Funnily it was working on NVIDIA hardware, even though the specification says otherwise, and I was wondering why I my app crashes on ATI cards.

Anyway, display lists are gone, but we need some complex state objects that could fill those holes that were left after them.

More callbacks

I was very happy to see the appearance of an extension that introduced the callback concept into OpenGL (GL_AMD_debug_output). Since that, the functionality was promoted to an ARB extension meaning that the ARB has accepted the fact that we need callbacks.

What I would like to see in the future is more OpenGL callbacks. One of the most trivial things I can think of are asynchronous queries. It would so much easier if we would be able to receive a callback from OpenGL when the results of our asynchronous queries are available, rather than having to manually poll it for result in various phases of the rendering.

Actually, I could imagine callbacks for every rendering command issued that will be called by the driver as soon as the actual rendering is complete on the GPU side.

Programmable blending

This is one another thing that developers are screaming for. Fortunately now we have indirect methods to solve most of the issues of programmable blending via the extensions GL_EXT_shader_image_load_store and GL_NV_texture_barrier, however a more general solution would be welcomed.

I don’t know whether this would be actually possible on current hardware but if not, then this is a message to hardware vendors to solve the issue in the near future.

Summary

We’ve seen that even though OpenGL is on track and the Khronos Group is keeping up the pace with its competitors, still there are lots of room for improvement regarding to the OpenGL specification from both functional point of view as well as from API design point of view.

I would like to end the article with a summary of what I expect to be part of the OpenGL 4.2 specification and my personal wish-list beyond those in some kind of priority order.

My expectations for OpenGL 4.2:

• GL_EXT_shader_image_load_store
• GL_ARB_shader_atomic_counters
• GL_ARB_instanced_arrays2
• GL_ARB_explicit_sampler_location
• GL_ARB_explicit_uniform_block_index

My personal wish-list for OpenGL 4.2:

• GL_ARB_draw_indirect2
• GL_ARB_direct_state_access
• GL_NV_texture_barrier
• GL_AMD_conservative_depth
• GL_ARB_texture_gather_lod
• GL_NV_copy_image
• GL_EXT_texture_filter_anisotropic
• GL_ARB_shader_stencil_export
• GL_AMD_depth_clamp_separate
• GL_AMD_transform_feedback3_lines_triangles