Many things have changed since the first time the public put their hands on the first mobile phone device as these days the end user rarely makes their choices when buying a mobile equipment based on their telephony capabilities. In fact, nowadays these devices are one of the most popular entertainment platforms out there. The main problem for application developers is that these platforms tended to be very heterogeneous from point of view of hardware architecture as well as that of API support. Meanwhile things have changed. While the underlying hardware still varies a lot from device to device the work of application developers has been eased by having cross platform mobile operating systems and open standards. In particular OpenGL ES that is an embedded version of the popular graphics API. In this article I would like to talk about some of the big players of the mobile OS industry and about using OpenGL ES for creating impressive mobile applications.

The first version of the OpenGL ES specification has been released in order to provide a lightweight API for embedded graphics using a well-defined subset of the functionalities provided by the desktop version of OpenGL. While the specification is already out quite for a while, the wide adoption in the industry and the interest of application developers for it became strong only in the recent past. Currently, we have several mobile platforms that are bundled with 3D accelerators and provide a set of features via OpenGL ES that makes developers capable of creating games that weren’t possible even on desktop platforms about ten years ago.

Going 3D on mobiles

Those who know me, know that well that I was always interested in graphics, especially when using it for entertainment purposes. In particular, I was about to develop video games since the first time I’ve put my hands on a computer. This is no different now as well as now I’m writing about OpenGL ES and mobile platforms because I got interested in creating games for mobile phones.

As I’ve already mentioned before, the problem with developing for mobile equipments is the variety of hardware and software platforms that they are built on. As being somebody who is already familiar with desktop OpenGL, having OpenGL ES in the tool-set already eliminates some of the burden that I must face with.

Also when talking about application platform things have also changed a lot. Nowadays, we have just a few big players in the mobile OS industry thus easing the work of the developers. More precisely, if an application developer plans to go mobile and would like to grab the biggest market audience, can limit their efforts on the following platforms:

• iPhone OS – This is the one that drives Apple’s iPhone mobile devices as well as the iPod Touch. It provides an application platform similar to that Mac developers got used to. It can be said that this platform is the most popular in the industry, especially when dealing with gaming applications.
• Android – This is the newest player in the field, brought by Google. While it’s a newbie in the industry it already captured the attention of tons of developers. We can say that currently Android and iPhone are dictating the direction of mobile entertainment.
• Symbian OS – Symbian has the largest share in most markets worldwide, still not that popular in the mobile gaming industry. It is the operating system running most of today’s Nokia phones.
• Windows Mobile – Microsoft’s product built on Windows CE, the company’s embedded operating system.
• RIM Blackberry OS – Operating system primarily designed for the business industry.

While most of these mobile operating systems are built on the same design conceptions it is very difficult for the developer to create cross-platform applications for all these platforms as they vary on the language and tool-set support that minimizes the possibilities for code reuse. Unfortunately this is against the one of the most important rule of mobile development as to maximize portability.

It is not 100% true that there is no way to provide optimum portability for all these platforms, but if we choose this direction we are limited to two possibilities: cross-platform Java applications and web-based applications. While these seem to be excellent alternatives to native programming of the platforms, they severely limit the developer in creating applications that fully take advantage of the underlying hardware. This is when OpenGL ES comes into picture as all these platforms have API support thus providing at least some form of code reuse possibility when dealing with entertainment applications.

Now, I would like to continue with talking about the two platforms that I’m most interested in.

iPhone OS

I started to get involved in iPhone game development because one of my friends pushed me to after seeing the great success of his brother-in-law, zhooley who had some great titles. Currently I don’t have a Mac yet to develop on, but already read some stuff about iPhone development. This is where the following information come from.

iPhone is currently is the most important platform for mobile application developers. It became such an important factor in the industry thanks to Apple’s AppStore. Previously there was little to no way for the end users to extend their mobile software base so easily. While this is good for the end user, it is maybe even better for application developers as AppStore provides them quite a large market audience.

The secret why iPhone is an excellent gaming platform lies in the palette of features that the phone hardware and the software frameworks provide. Just to mention the most important ones:

• Touch screen control with support for multi-touch events capturing the movement of up to five fingers.
• Three accelerometers for tracking the spacial movement and direction of the device in all axes.
• MVC inspired GUI framework for enhanced productivity.
• Support for several industry standard APIs like OpenGL ES, OpenAL and much more.

But that’s enough from the general speaking, let’s see what’s about OpenGL ES support on the iPhones…

As far as I can tell, not being an iPhone owner, the graphics hardware bundled with the mobile comes in form of PowerVR accelerators: MBX and SGX.

The PowerVR MBX has OpenGL ES 1.1 support, that is roughly equivalent to OpenGL 1.5, running a tile-based deferred renderer that is suitable for most 3D applications. That means it has only fixed function capabilities, however that is usually enough for most mobile applications. Also note that it has very limited amount of texture memory of 24MB.

The PowerVR SGX is a more powerful processor that also supports OpenGL ES 2.0, roughly equivalent to OpenGL 2.0, but has optimized fixed function shaders that provide flawless backward compatibility for OpenGL ES 1.1 applications.

The most important thing is still that all iPhones are able to do floating point maths natively and efficiently that is an important factor when dealing with OpenGL applications as the usage of the fixed point types can be quite a burden for developers, especially for those migrating from desktop development.

Additionally, the OpenGL ES implementation on iPhone provides some nice extensions like GL_OES_framebuffer_object, GL_OES_compressed_paletted_texture and GL_OES_point_sprite. Also, thanks to the iPhone simulator that comes with the SDK it is easy to test the application during development without an actual device. Still, one important hint to mention is that the iPhone simulator has different OpenGL ES capabilities than the actual hardwares and also the performance characteristics measured on the simulator should not be taken as valid measurements because the simulator does not really simulate the graphics hardware but only the software platform.

iPhone development is done using the Cocoa API and preferably Objective-C, however C, C++ and Objective-C++ can be also used for development. One just has to interface somehow the Cocoa API and the rest can be done almost in any native programming language. That is one of the key advantages of the iPhone platform that one can develop native applications and no need for Java or web-based solutions.

While iPhone may seem to be a perfect choice for mobile game platform, we should not forget about one big disadvantage of it, in particular that one cannot develop legal iPhone applications without owning a Mac.

Android

The Android platform was suggested by one of my workmates who just brought a Droid. That phone is actually a device capable to compete with the iPhone from both features and performance point of view.

Android is the big hit of the last year and my forecast is that it will be one of the most relevant platforms of the upcoming years. Google adopted the idea of Apple and they also created an open market for the softwares that the end user can easily download and install on their devices. This is the AndroidMarket that can easily become a powerful competitor of the AppStore.

While, as I said earlier, the Motorola Droid, as an example, does support about the same feature set that makes the iPhone an excellent gaming platform, this cannot be said about most of the phones running Android on them. This is maybe one of the biggest disadvantages of the Android platform. However, we can take this also as an advantage as it makes it possible for more phones to adopt this operating system.

As the Android operating system is running on various phones from different vendors with different hardware capabilities, there isn’t too much to talk about the graphics hardware capabilities except that some devices not just don’t have a graphics accelerator but they also lack of floating point support. This is another disadvantage as it forces developers to stick to fixed point math in their OpenGL ES applications to maximize portability or they have to maintain two different rendering paths.

Originally, Android supported only OpenGL ES 1.0 that is roughly equivalent to OpenGL 1.3. However, since NDK r3 there is also OpenGL ES 2.0 support for Android as well. The feature set here varies much more from both hardware point of view and extension support.

Development for Android is done in Java using a proprietary SDK for accessing the Android API. The SDK comes with a simulator that works fine, except the long initial boot time that I was really surprised about when first trying it out.

One advantage of the SDK that it can be used in virtually any operating system so application developers can work on either Windows, Linux, MacOSX or other platform. There is also a nice Eclipse plugin that makes application development for Android even easier. That’s why I started with this one.

Just to illustrate how easy to put together some working demo with a good SDK, I’ve created a simple box rotating app to demonstrate OpenGL ES usage on Android. From installation till having a working application it took no more than two hours. You can find the download links for both the source code and the binary release at the end of the article.

Why mobile games?

I am a person who was, is and will be interested in developing computer games. Previously, I was working with desktop platforms and at the time when I was 10 years old it was satisfactory to put together some simple 2D game but not now.

I had always planned to create a state-of-the-art game engine and use it for some game, like most people like me do, but the efforts of one is simply unsatisfactory to compete with the players in the industry out there. Even if I feel the capability to be able to write such an engine but it would take that much time that I simply don’t have since I am working. Even if I would manage to accomplish it in a year or two then the problem with content creation comes into picture. For an AAA PC game content creation takes several times more than the actual programming and here I even lack the knowledge to achieve it. On the other hand mobile game creation is a much shorter process when you can get to actual results in a matter of weeks that is far better compared to PC game creation.

Also, I would never use third party game engines, except some basic libraries like OpenGL, a physics library and things like that because otherwise I wouldn’t feel the results being my own creation.

Having game development as a hobby works well during high school and university but it gets quite difficult after you are out there in the world having a job and responsibilities. Maybe I should have been already taking my time before to develop something concrete for PC but, as most fellow hobbyist know, you usually end up having hundreds of unfinished projects.

While I would never forget about desktop platforms and I will actively keep myself up with the evolution of the industry, mobile application development opened another world for me where I can unfold myself.

HelloAndroid Demo

Source code: helloandroid_src.zip
Binary release: HelloAndroid.apk

There was always big need for libraries that provide an abstract interface towards the basic platform specific facilities that are necessary for setting up an execution environment for a particular application. In the OpenGL world one of the first such libraries was GLUT. After a while more and more functionalities were put into these libraries that reflect more or less the requirements of application developers. One such framework is SDL. It seems that SDL is still the most respected one of these and it is preferred by the developer community. However, in this topic I will present an alternative that proved its superiority to me in the last few months…

Why would I need such a library?

There are loads of reasons why it is good to have such a framework in your toolkit. I would like to present only a few that I consider important. First of all, having some easy to use API to setup the basic environment for your application, like a window with an OpenGL rendering context, simply removes the burden from dealing with such platform specific details and concentrate on the actual product. Next, they usually give a quite good degree of platform portability so you don’t have to study specific operating system APIs and you can still deploy your application on multiple platforms. I can recite many other reasons but the most important one is that they are reusable components so you don’t reinvent the wheel.

For a while I was quite satisfied with my own implementation of such a toolkit until I moved from Delphi to C++ development. This forced me to look around on the market to find a replacement for my proprietary solution as C++ has a very large developer community so it shouldn’t be that hard to find a suitable framework. Well, actually this wasn’t the case as it was the time when the OpenGL 3 specification came out with its new context creation and deprecation model. It was very embarrassing to observe that even the most popular multimedia frameworks are hardly adopting these new features.

At that time I realized that my library choice will heavily affect my productivity in the future so I have to think well which one I will use afterwards. To ease the selection I created something like a wish-list about what I expect from such a library. The most important issues were the following:

• Feature-rich - The library must provide the most basic functionalities needed for an average OpenGL application. This includes window and rendering context handling, keyboard and mouse user input capture, timing facilities and, of course, supports OpenGL 3 contexts. Optionally it would be nice if the API also provides multi-threading, image and audio handling, basic network operations, joystick support, etc.
• Modular - The library must be modular, that means I can select which components of the framework I would like to use in a particular application. Sometimes less is more so, as an example, in a very simple cube rotating OpenGL demo I don’t want to link against a library which contains network handling. Such monolithic libraries makes life much harder as they usually rely on exotic dependencies and prevent easy deployment of the application.
• Portable - It should be portable, at least it should work on the three most popular PC platforms: Windows, Linux and MacOSX. It has to work also with a variety of build systems, preferably with Visual C++, GCC and Xcode. Optionally it would be nice if it can be interfaced by applications written in other programming languages than C/C++.
• Easy to use - In an ideal world such a framework should have a very clean interface and its usage must be very natural for the developer. This issue is however rather subjective so what it easy to use for one developer maybe it’s not the best for another. Regarding to this issue I will present the alternatives, obviously, from my perspective.

Maybe choosing such a multimedia library for an OpenGL hobby project is just a matter of taste, but when it comes to support for OpenGL 3 context support, developers have very limited choices and one most probably faces decisions when they have to make some trade-offs when selecting any of  these libraries. Anyway, as one of my key requirements is that the library must support OpenGL 3 contexts, it is not that difficult to present all the most popular alternatives.

Simple DirectMedia Layer

SDL has a long history in this domain and it proved that it is an excellent choice for most hobbyists and even for professionals. It has been used in tons of different free and commercial applications and it is probably still the most preferred library in this category. Lets examine it regarding to the issues presented previously.

SDL provides almost all the facilities that are needed for an average graphics application. Together with some additional libraries developed as an extension to SDL like SDL_image and SDL_net it conforms to almost all of my requirements regarding to feature content. The most important is that its latest development branch also has support for OpenGL 3.

From point of view of modularity, especially taking into account that the less common used facilities are provided by different add-ons, SDL seems to be a good choice. However, the SDL core itself has already a bit too much dependencies on other operating system specific libraries, especially the reliance on DirectX on the Windows platform. Even if probably most of you can live with this, it is simply unacceptable for me. Anyway, this is still not the biggest problem what I’ve faced with when I checked out whether SDL suites my needs.

Platform portability is one of the key advantages of SDL as it even supports many other platforms than what was on my wish-list. Also regarding to language portability, the C interface of SDL makes it very easy to drop into a Delphi project as an example. Actually there are also plenty of  bindings for other languages for interfacing SDL including but not limited to Java, C#, Delphi, Ada, Perl and python. However, when it comes to build system portability I have to mention my bad experiences.

First of all I am that kind of animal who uses GCC also for compilations under Windows. As SDL comes with an automake based build system which proved to be unusable using MSYS and I would rather not use Cygwin as it also introduces quite many unwanted external dependencies. After giving up compilation using MinGW, I tried to build the library using the good old Visual C++ IDE. This is when I faced the problem that I would have to install the DirectX SDK in order to compile SDL. The final hit was that even after downloading and installing the huge DirectX SDK, SDL still refused to compile with weird compiler errors.

By the way, all these compilation related issues wouldn’t be a problem for me if SDL would come with a binary release. Even if it has such releases for earlier versions, it does not have it for the latest development branch which contains the OpenGL 3 related stuff. So actually I cannot even prove that the OpenGL 3 specific implementation in SDL works in practice or not.

The API interface provided by SDL got skewed up meanwhile, arising from the fact that SDL has a long history, but still I cannot say that the interface is not clean enough to be easily used in any project. So in this regard SDL is still a major player.

The OpenGL Framework

My second was GLFW as it was the only library that supported OpenGL 3 as far as I knew at that time. For those who are not familiar with this library, GLFW is a simple yet powerful toolkit with a similar interface like GLUT but with added capabilities like multi-threading and joystick support (see GLFW home page).

It is a very feature-rich framework compared to its size and even not being very modular, it still does not involve that much external dependencies as SDL. As it also provides OpenGL 3 support only in it’s latest development branch I had to compile the code here as well, however, it was straightforward to do it even using MinGW so I don’t have any complains regarding to this subject. Also it supports multiple platforms, at least those that I am mainly interested in.

Unfortunately, as many other early OpenGL 3 context handling implementations, it wasn’t working on all platforms. More precisely, under Windows the OpenGL 3 code in the development at that time (about half year ago) worked only with NVIDIA cards as, non-compatible with the Microsoft WGL specification, NVIDIA’s ICD exposed the wglCreateContextAttribARB function even if there was no valid OpenGL context bound and, as usual, the developers used only NVIDIA cards for testing which resulted in a partially working OpenGL 3 context handling implementation.

As the code of GLFW is also very straightforward and handy to read, I easily corrected the bug in the OpenGL 3 context handling and I used GLFW for a few months for my hobby projects. After a while, however, the lack of some facilities in GLFW made me to think through this library selection again as I didn’t want to end up using several different libraries for different purposes which would result in a barely well designed code structure.

Simple and Fast Multimedia Library

We’ve arrived to the main topic of this article. Finally I’ve found SFML which at first sight seemed to be “just another multimedia library” but I soon realized that it’s far more than that.

First of all, it has all the features I needed. Beside the basic ones, it has very nice support for networking but not just basic socket programming using TCP or UDP packets, but a more comprehensive toolkit for even HTTP and FTP transactions. It also has built-in sound support via OpenAL which was another thing that caught my attention as I also preferred OpenAL over other audio libraries like fmod. Beside this, it has many other interesting features but you’d better check out the SFML project site.

As I already got used to, SFML had also support for OpenGL 3 context handling but only in the latest development branch. Anyway, this seemed to be no problem as I had no building issues like that what I met in case of SDL. What is more important that the OpenGL 3 implementation actually worked flawlessly, there was no need for any modification. Just to demonstrate, setting up an OpenGL 3 window with SFML can be as easy as writing the following line of code:

sf::Window App(sf::VideoMode(800, 600, 32), "OpenGL 3 window", sf::Style::Close, sf::ContextSettings(24, 8, 0, 3, 2));

When it comes to modularity, SFML also wins at me. First, it does not need any exotic libraries or headers for rebuilding the framework. Beside this, it has minimal number of external dependencies but only to the operating system libraries used. It is also modular as different sub-systems of the framework are compiled into different library modules so in your project you can simply select which ones do you intend to use to further minimize deployment issues.

From portability point of view, SFML supports all the platforms that are important for me. Also SFML works with no fuss indifferent with all the compiler tool-chains I’ve tried. At first sight I had concerns regarding to the language portability of SFML as is was written in C++ and this C++ interface is exposed to the client. However, this issue was solved by the library by providing a C wrapper called CSFML together with the framework itself which makes it rather straightforward to write binding for virtually any programming language.

Conclusion

If I haven’t convinced you so far that SFML can be the perfect choice as a multimedia library for almost any hobby or commercial product then you should check out the full feature list and see for yourself. I will probably write more about SFML in the future and you will also meet some usage examples in my upcoming demos.

If you are not interested in the additional features provided by SFML, instead you are just searching for a very basic framework that provides you a window and some user input handling then GLFW can be your choice. However, based on my bad experiences I would not advise anymore the usage of SDL to anybody but maybe I’m a bit too inclement.