The Khronos Group keeps the pace that they set themselves being able to deliver the latest specification of OpenGL less than half year after the revolutionary appearance of OpenGL 4. Abandoning the OpenGL 3.x line of the specification (at least for a while) the new update concentrates on Shader Model 5.0 class GPUs and extensions heavily promoted by the community. Beside all this, the Khronos Group now confessedly opens towards convergence to OpenGL ES making the desktop version of the specification downward compatible with its embedded brother. In this article I would like to present the features introduced with the latest revision of the specification.

At the time of the release of the OpenGL 4 specification I was able to quickly deliver you a thorough presentation of all the new features introduced by that revision of the specification. This time I am already quite late, however I hope that this article will still prove as value for lots of you, especially for those who haven’t had time in the recent past to dig into the details of the new API version.

OpenGL 4.1 is not as revolutionary and feature-rich as its predecessor, however the latest revision was well received by the community as it brought such core extensions to the API that the community was waiting for a long time now. The new revision of the specification was accompanied with the appearance of a couple of other ARB extensions that have not yet been included into core, however I will still talk about some of them as they indicate a slight shift in the force of influence of various vendors and representatives inside the Architecture Review Board (ARB).

New features of OpenGL 4.1

Let’s start with the presentation of the new features arriving with the OpenGL 4.1 specification primarily targeting Shader Model 5.0 hardware. Here you will see a lot of harmonization features as well as community’s choice features that squarely intended to increase OpenGL development efficiency and feedom.

ARB_ES2_compatibility

There have been for a long time rumors about the Khronos Group preparing a convergence between desktop OpenGL and OpenGL ES. This extension of the core specification clearly makes the first step towards this goal by providing an all-in-one specification pack that makes the desktop version of the specification downward compatible with ES. The extension adds support for features of OpenGL ES 2.0 that are missing from OpenGL 3+. According to the extension specification, enabling these features will ease the process of porting applications from OpenGL ES 2.0 to OpenGL.

More precisely, GL_ARB_ES2_compatibility exposes not just all the functions and tokens that weren’t present in the desktop version of the specification but also completes it with all the semantics that were exclusively specified only in the embedded version. Just to mention few of these issues:

• Vertex data format is now extended with the possibility to use 16-bit fixed point values by exposing the GL_FIXED type identifier token.
• Providing possibility to query the precision format used internally by shaders.
• Enable the use of GLSL ES for writing shaders for desktop GL.

While having this extension under the hood does not mean that we can simply pick our last game made for e.g. Symbian and just drop it on our PC, this extension may prove to be great value for GL ES developers migrating their software to desktop platforms.

ARB_get_program_binary

This is one of the most waited additions to the core specification by the developer community. This extension introduces the possibility to acquire some sort of binary format of the compiled and linked shaders that can be later used to specify the program object directly with its binary code thus providing caching possibility to eliminate the need of compilation and linking next time the shader has to be used. This also makes it possible to create an offline GLSL compiler just using the OpenGL API itself.

Still, it has to be mentioned that having this feature in our hand does not necessarily mean that we can simply create our shader binaries offline and then distribute our software without the shader source itself as the binary formats supported by a particular implementation heavily depend on the hardware vendor as well as driver version. This is due to the fact that the shader binary most probably consists of instructions specially generated for the particular GPU-driver combo. The only way to relax this limitation would be to have some sort of cross-platform byte-code for shaders but that would in fact defeat most of the benefits of the extension on its own. Additionally, this extension does not provide any binary formats but leaves this to vendor specific extensions. It only exposes a common infrastructure for acquiring and loading program binaries.

While the usage of this extension does not completely eliminates the need for shader source compilation, it can limit the need for recompilation and relink to an installation time or first-run time compilation instead and use the stored binaries later. It also opens up room for SDK tools providing shader compilers with more aggressive optimization at their disposal being used offline. Such tools can truly be introduced as the specification explicitly mentions that run-time generated binaries by the GL should be interchangeable with those generated by offline SDK tools.

ARB_separate_shader_objects

This is one another extension requested over several forums by the community. This feature has a longer history as it is actually based on the already existing and widely supported extension GL_EXT_separate_shader_objects by NVIDIA. For those who are already familiar with the predecessor of this extension won’t really find too much new stuff reading the specification of the ARB version of the extension, however it is still a must to read for them as well as even though there aren’t too much semantic differences between the functionality of the two, the usage of them still differs quite a lot as the ARB version solved the design issues of its predecessor by introducing a new type of GL object that I will talk about just in a moment.

In a nutshell, this extension provides a way to create program objects using any variation of shaders and bind them together to the current rendering context. Previously there was no way to bind multiple program objects to the context as the program object was designed to be a container for all the shaders forming the rendering pipeline of the context. This was a design decision during the development of GLSL that, before this extension, made the connection between the varyings of subsequent shader stages using a name based binding. As name information is available for shaders latest in the link stage, shaders were tightly coupled meaning that a change in any shader stage code required the relinking of the complete program object.

This proved to be very unpleasant for OpenGL developers as usually every rendering engine has its own set of vertex and fragment shaders (maybe accompanied with other shader types) that are used in various combinations. As an example, let’s take two vertex shaders: a simple MVP matrix based transformation shader and a more complex one that also supports skeletal animation. Also let’s take two fragment shaders: one for diffuse material and one for reflective material. We can have several types of objects: static with diffuse material, static with reflective material, animated with diffuse material and animated with reflective material.

In traditional GLSL the vertex and fragment shaders are bound together at link time rather than at the time they are bound to the context, like it was in case of legacy shaders (GL_ARB_vertex_program, GL_ARB_fragment_program and others). This means that in order to be able to use any of the combinations of vertex and fragment shaders (and maybe some geometry and tesselation shaders as well) we end up with two possible solutions, both having their severe drawbacks:

Link every combination of the shader objects

While this sounds as a viable solution and is still used by most of the developers, it has several problems. First of all, it wastes resources as we now have several copies of the same piece of code and the number of combinations can be pretty high, especially if not just vertex and fragment shaders are in use. While this is already quite a reasonable issue with the solution, the biggest problem arises for the application developer when he or she has to maintain an individual set of uniform locations as well as binding points for vertex attributes, draw buffers and possibly transform feedback buffers. While the GL_ARB_explicit_attrib_location extension already eliminates the need for maintaining binding points for vertex attributes, this solution is still simply unacceptable.

Link the program objects on an on-demand basis

In case of this alternative we are said to link the shader objects only when they are actually needed. While this solution eliminates the need for a possibly huge number of program objects, it introduces a reasonable run-time performance hit due to the additional relink process needed. Additionally, this solution proves to be more inferior even compared to the previous one as the uniform locations are determined at link time so it makes no less headache to the application developer.

This is the rationale behind this extension and why it is included into the core specification. The extension relaxes the strict tightly coupled behavior of the GLSL and adopts a mix-and-match shader stage model allowing multiple different program objects to be bound at once each to an individual set of rendering pipeline stage independently of other stage bindings.

Due to the fact that from now program objects are not the top most containers for the code used currently by the rendering pipeline, the ARB decided to introduce a new container object called a “program pipeline object” that can contain a set of program objects bound to their very own set of shader stages. This is the main difference between the EXT and the ARB version of the extension. I think it was a good decision to introduce this new type of object and the associated semantics as I always thought that the EXT version of the extension doesn’t have a really good design as I’ve seen it kind of a hack to relax the limitations of GLSL. The program pipeline object idea is definitely superior and I hope that the GLSL does not have too much of such annoying design issues hidden within.

ARB_shader_precision

This extension is much more a clarification to the existing specification rather than a new feature. It restricts more clearly the precision requirements of implementations of GLSL. According to the specification, the extension is meant to more precisely define the precision of arithmetic operations (addition, multiplication, etc.), transcendentals (log, exp, pow, etc.), when NaNs (not-a-number) and INFs (infinites) will be accepted and generated and denorm flushing behavior. The precision of the rest of the operations, including trigonometric operations are not addressed by the extension. For further details, please refer to the extension specification.

ARB_vertex_attrib_64bit

This extension trivially introduces 64-bit floating-point types into the list of supported vertex attribute component types. Nominally OpenGL did support this already from the very early stages of its history, however in practice only the latest generation of hardware does really accept vertex attributes in double precision floating-point type. While OpenGL 4 already introduced support for 64-bit floating-point values in GLSL and most of the shaders’ environment, vertex attributes gained the 64-bit precision only with this new extension.

This new feature makes it possible to use high precision for positioning data and other attributes of our geometries. While this sounds pretty awesome and it is actually, still for game developers and other real-time graphics users this shouldn’t mean that they should quickly switch to the new precision only in such cases when the precision requirements of the application really need it as using 64-bit floating-point values for vertex attributes does not just double the memory consumption but also involves a serious hit on performance due to bandwidth limitations and vertex attributes of this type may count double against the implementation-dependent limit on the number of vertex shader attribute vectors.

ARB_viewport_array

Previously, the configuration of the viewport, aka the transformation that generates the screen space coordinates based on the incoming view space coordinates of the vertices, was a global configuration that had effect on all draw commands meaning that in order to draw a primitive into multiple viewports the OpenGL viewport had to be changed between several draw calls. While previously this limitation wasn’t really an issue, due to the introduction of geometry shaders the possibility to amplify geometry and produce multiple output primitives for each primitive input justifies the need of several separately configurable viewports. Why? Because even though one was able to render the output primitives into separate render targets, they still shared the same global viewport.

This extension enhances OpenGL by providing a mechanism to specify multiple viewports and a new ability for the geometry shader being able to select the used viewport on a per-primitive basis. This does not just mean that separate viewports can be used for separate render targets but also enables to use multiple viewports to render to the same render target.

Additionally, the introduction of a viewport array means that we’re gonna have separate scissor rectangle for each viewport in the array as well. This can come handy for deferred shading based renderers that often use the scissor rectangle to limit the number of pixels to be accessed in case of rendering the effect of a light source. Having multiple scissors means that we have to change state less often, thus batching is much less an issue even in case of heavy scissor rectangle usage.

Finally, the new viewport specification commands accept floating point values thus providing additional flexibility to the application developer to define their very own pixel center conventions.

I’m pretty unsure whether this feature depends on any Shader Model 5.0 hardware, maybe others are more aware of this. Anyway, I wouldn’t be surprised if this extension will be supported by a much larger range of graphics cards than just pure SM5 GPUs. Actually this is true for many other extensions introduced by OpenGL 4.1 but let’s not guess but wait for the upcoming drivers to see whether I’m right or wrong.

Some other interesting extensions

So far I presented the new features of the latest revision of the OpenGL specification. While this was the main topic of this article, at about the same time the specification was published, a lot of other ARB extensions just appeared in the registry. While these extensions are not yet included into core and I cannot know whether they will be ever included, I would like to talk about some of them as it made me get to an interesting conclusion.

ARB_shader_stencil_export

The stencil test is a powerful mechanism of OpenGL to selectively discard fragments based on the content of the stencil buffer that is used in a wide variety of rendering techniques including shadow volumes and deferred shading. However, the whole configuration of the stencil test and stencil operations is completely fixed function that is limited to operations such as incrementing, decrementing the existing value, or replacing the existing value in the stencil buffer with a fixed reference value.

This extension provides some programmability to the fixed function stencil operations by enabling the fragment shader to output a stencil reference value on a per-fragment basis. When stencil testing is enabled, this allows the test to be performed against the value generated in the shader. Also, when the stencil operation is set to GL_REPLACE, this allows a value generated in the shader to be written to the stencil buffer directly.

This opens up a lot of possibilities, however, I need to think much more about it as the best use cases of this feature are pretty much not basic ones. Obviously, by using the stencil reference value export inside a fragment shader disables early stencil test in the same style as exporting an new depth value from within a fragment shader disables early depth test.

ARB_debug_output

This extension allows OpenGL to notify the application when various events occur that can come handy during application development and debugging. These events include errors, usage of deprecated functionalities, using configuration that results in undefined behavior, portability or performance issues. The application is notified about these events using a callback function that is defined by passing a function pointer to the appropriate OpenGL command.

While this extension provides a callback mechanism only for debugging purposes, the most revolutionary thing by having such an ARB extension is that this is the first official appearance of a feature that supports callbacks to the application code. Most probably not I’m the only person who would like to see a lot of other callbacks in the future included in the OpenGL API as we can benefit from it by getting notification about e.g. the completion of various asynchronous commands issued previously. This does not just provide a lot of flexibility but may also help in optimizing the rendering code based on the additional information previously available only if we use polling.

Why these extensions are so interesting?

The two extensions presented above already great value on their own but this isn’t why I mentioned them. The reason why I found these extensions so interesting as they are both obviously based on some vendor specific extensions released in the recent past by AMD, namely GL_AMD_shader_stencil_export and GL_AMD_debug_output. This conspicuously reveals that AMD has serious plans with their OpenGL support and this is something that a lot of those crazy folks waited for, who develop OpenGL stuff using ATI cards like me.

I think this also means that the NVIDIA monopoly in the ARB is over and this results in concurency and competition from what OpenGL and its community will definitely benefit in the long run.

Conclusion

The article ran out of control again, like the one I wrote about the previous release of the specification. Again, hope there are at least a few of you who kept up reading and finally got to this last chapter of the article. We can again quote the always recurring question of the community:

Where is direct state access?

Well, it is still not here, however, finally AMD has finished implementing it as well and published it finally. They have been working on it for quite some time but it became officially public only with Catalyst 10.7. Haven’t used it so far so maybe plenty of hidden bugs are still in it but at least they have it. This is one another thing that strengthens my prognostication that AMD committed itself for support OpenGL as previously they barely added support for any other extensions beside core features.

Back to the topic of the OpenGL 4.1 specification, while it is not as revolutionary as we got used to after reading the previous update, OpenGL is still on track and this is thanks to the Khronos Group and obviously to the great community. If OpenGL will get its iterative evolution in this pace like we’ve seen in the last two years, Microsoft will have a difficult time to keep up.

Thanks for reading this not-so-short article!

Many things have changed since the first time the public put their hands on the first mobile phone device as these days the end user rarely makes their choices when buying a mobile equipment based on their telephony capabilities. In fact, nowadays these devices are one of the most popular entertainment platforms out there. The main problem for application developers is that these platforms tended to be very heterogeneous from point of view of hardware architecture as well as that of API support. Meanwhile things have changed. While the underlying hardware still varies a lot from device to device the work of application developers has been eased by having cross platform mobile operating systems and open standards. In particular OpenGL ES that is an embedded version of the popular graphics API. In this article I would like to talk about some of the big players of the mobile OS industry and about using OpenGL ES for creating impressive mobile applications.

The first version of the OpenGL ES specification has been released in order to provide a lightweight API for embedded graphics using a well-defined subset of the functionalities provided by the desktop version of OpenGL. While the specification is already out quite for a while, the wide adoption in the industry and the interest of application developers for it became strong only in the recent past. Currently, we have several mobile platforms that are bundled with 3D accelerators and provide a set of features via OpenGL ES that makes developers capable of creating games that weren’t possible even on desktop platforms about ten years ago.

Going 3D on mobiles

Those who know me, know that well that I was always interested in graphics, especially when using it for entertainment purposes. In particular, I was about to develop video games since the first time I’ve put my hands on a computer. This is no different now as well as now I’m writing about OpenGL ES and mobile platforms because I got interested in creating games for mobile phones.

As I’ve already mentioned before, the problem with developing for mobile equipments is the variety of hardware and software platforms that they are built on. As being somebody who is already familiar with desktop OpenGL, having OpenGL ES in the tool-set already eliminates some of the burden that I must face with.

Also when talking about application platform things have also changed a lot. Nowadays, we have just a few big players in the mobile OS industry thus easing the work of the developers. More precisely, if an application developer plans to go mobile and would like to grab the biggest market audience, can limit their efforts on the following platforms:

• iPhone OS – This is the one that drives Apple’s iPhone mobile devices as well as the iPod Touch. It provides an application platform similar to that Mac developers got used to. It can be said that this platform is the most popular in the industry, especially when dealing with gaming applications.
• Android – This is the newest player in the field, brought by Google. While it’s a newbie in the industry it already captured the attention of tons of developers. We can say that currently Android and iPhone are dictating the direction of mobile entertainment.
• Symbian OS – Symbian has the largest share in most markets worldwide, still not that popular in the mobile gaming industry. It is the operating system running most of today’s Nokia phones.
• Windows Mobile – Microsoft’s product built on Windows CE, the company’s embedded operating system.
• RIM Blackberry OS – Operating system primarily designed for the business industry.

While most of these mobile operating systems are built on the same design conceptions it is very difficult for the developer to create cross-platform applications for all these platforms as they vary on the language and tool-set support that minimizes the possibilities for code reuse. Unfortunately this is against the one of the most important rule of mobile development as to maximize portability.

It is not 100% true that there is no way to provide optimum portability for all these platforms, but if we choose this direction we are limited to two possibilities: cross-platform Java applications and web-based applications. While these seem to be excellent alternatives to native programming of the platforms, they severely limit the developer in creating applications that fully take advantage of the underlying hardware. This is when OpenGL ES comes into picture as all these platforms have API support thus providing at least some form of code reuse possibility when dealing with entertainment applications.

Now, I would like to continue with talking about the two platforms that I’m most interested in.

iPhone OS

I started to get involved in iPhone game development because one of my friends pushed me to after seeing the great success of his brother-in-law, zhooley who had some great titles. Currently I don’t have a Mac yet to develop on, but already read some stuff about iPhone development. This is where the following information come from.

iPhone is currently is the most important platform for mobile application developers. It became such an important factor in the industry thanks to Apple’s AppStore. Previously there was little to no way for the end users to extend their mobile software base so easily. While this is good for the end user, it is maybe even better for application developers as AppStore provides them quite a large market audience.

The secret why iPhone is an excellent gaming platform lies in the palette of features that the phone hardware and the software frameworks provide. Just to mention the most important ones:

• Touch screen control with support for multi-touch events capturing the movement of up to five fingers.
• Three accelerometers for tracking the spacial movement and direction of the device in all axes.
• MVC inspired GUI framework for enhanced productivity.
• Support for several industry standard APIs like OpenGL ES, OpenAL and much more.

But that’s enough from the general speaking, let’s see what’s about OpenGL ES support on the iPhones…

As far as I can tell, not being an iPhone owner, the graphics hardware bundled with the mobile comes in form of PowerVR accelerators: MBX and SGX.

The PowerVR MBX has OpenGL ES 1.1 support, that is roughly equivalent to OpenGL 1.5, running a tile-based deferred renderer that is suitable for most 3D applications. That means it has only fixed function capabilities, however that is usually enough for most mobile applications. Also note that it has very limited amount of texture memory of 24MB.

The PowerVR SGX is a more powerful processor that also supports OpenGL ES 2.0, roughly equivalent to OpenGL 2.0, but has optimized fixed function shaders that provide flawless backward compatibility for OpenGL ES 1.1 applications.

The most important thing is still that all iPhones are able to do floating point maths natively and efficiently that is an important factor when dealing with OpenGL applications as the usage of the fixed point types can be quite a burden for developers, especially for those migrating from desktop development.

Additionally, the OpenGL ES implementation on iPhone provides some nice extensions like GL_OES_framebuffer_object, GL_OES_compressed_paletted_texture and GL_OES_point_sprite. Also, thanks to the iPhone simulator that comes with the SDK it is easy to test the application during development without an actual device. Still, one important hint to mention is that the iPhone simulator has different OpenGL ES capabilities than the actual hardwares and also the performance characteristics measured on the simulator should not be taken as valid measurements because the simulator does not really simulate the graphics hardware but only the software platform.

iPhone development is done using the Cocoa API and preferably Objective-C, however C, C++ and Objective-C++ can be also used for development. One just has to interface somehow the Cocoa API and the rest can be done almost in any native programming language. That is one of the key advantages of the iPhone platform that one can develop native applications and no need for Java or web-based solutions.

While iPhone may seem to be a perfect choice for mobile game platform, we should not forget about one big disadvantage of it, in particular that one cannot develop legal iPhone applications without owning a Mac.

Android

The Android platform was suggested by one of my workmates who just brought a Droid. That phone is actually a device capable to compete with the iPhone from both features and performance point of view.

Android is the big hit of the last year and my forecast is that it will be one of the most relevant platforms of the upcoming years. Google adopted the idea of Apple and they also created an open market for the softwares that the end user can easily download and install on their devices. This is the AndroidMarket that can easily become a powerful competitor of the AppStore.

While, as I said earlier, the Motorola Droid, as an example, does support about the same feature set that makes the iPhone an excellent gaming platform, this cannot be said about most of the phones running Android on them. This is maybe one of the biggest disadvantages of the Android platform. However, we can take this also as an advantage as it makes it possible for more phones to adopt this operating system.

As the Android operating system is running on various phones from different vendors with different hardware capabilities, there isn’t too much to talk about the graphics hardware capabilities except that some devices not just don’t have a graphics accelerator but they also lack of floating point support. This is another disadvantage as it forces developers to stick to fixed point math in their OpenGL ES applications to maximize portability or they have to maintain two different rendering paths.

Originally, Android supported only OpenGL ES 1.0 that is roughly equivalent to OpenGL 1.3. However, since NDK r3 there is also OpenGL ES 2.0 support for Android as well. The feature set here varies much more from both hardware point of view and extension support.

Development for Android is done in Java using a proprietary SDK for accessing the Android API. The SDK comes with a simulator that works fine, except the long initial boot time that I was really surprised about when first trying it out.

One advantage of the SDK that it can be used in virtually any operating system so application developers can work on either Windows, Linux, MacOSX or other platform. There is also a nice Eclipse plugin that makes application development for Android even easier. That’s why I started with this one.

Just to illustrate how easy to put together some working demo with a good SDK, I’ve created a simple box rotating app to demonstrate OpenGL ES usage on Android. From installation till having a working application it took no more than two hours. You can find the download links for both the source code and the binary release at the end of the article.

Why mobile games?

I am a person who was, is and will be interested in developing computer games. Previously, I was working with desktop platforms and at the time when I was 10 years old it was satisfactory to put together some simple 2D game but not now.

I had always planned to create a state-of-the-art game engine and use it for some game, like most people like me do, but the efforts of one is simply unsatisfactory to compete with the players in the industry out there. Even if I feel the capability to be able to write such an engine but it would take that much time that I simply don’t have since I am working. Even if I would manage to accomplish it in a year or two then the problem with content creation comes into picture. For an AAA PC game content creation takes several times more than the actual programming and here I even lack the knowledge to achieve it. On the other hand mobile game creation is a much shorter process when you can get to actual results in a matter of weeks that is far better compared to PC game creation.

Also, I would never use third party game engines, except some basic libraries like OpenGL, a physics library and things like that because otherwise I wouldn’t feel the results being my own creation.

Having game development as a hobby works well during high school and university but it gets quite difficult after you are out there in the world having a job and responsibilities. Maybe I should have been already taking my time before to develop something concrete for PC but, as most fellow hobbyist know, you usually end up having hundreds of unfinished projects.

While I would never forget about desktop platforms and I will actively keep myself up with the evolution of the industry, mobile application development opened another world for me where I can unfold myself.

HelloAndroid Demo

Source code: helloandroid_src.zip
Binary release: HelloAndroid.apk