The “should happen” is the crux of the problem. Your Unit Testing is dictating the implementation. I would not create a VBO class or a Vertex Array class. I instead might have a Cube class that was using glBegin and now I want to change it to a vertex array.

This is just a bad way to do this.

Your example of switching from vertex arrays to VBOs should happen like the following:
- create the vertex array class and write the test cases for it to check correct OpenGL behavior
- think about how to introduce VBOs
- create abstract base class that vertex array and VBO inherit from
- create the VBO implementation with it’s own unit test set
- use the abstract base class in other modules as type and use factory pattern or whatever seems to be convenient
- simulate the vertex array/VBO functionality with mocks when testing other classes, so you don’t depend on any implementation details
Business logic shall always use abstract interfaces, not knowing the implementation details. For unit testing that particular vertex array/VBO class it is not a problem to have dependencies. Dependencies shall be avoided between modules and by using factory pattern and a mocking framework when doing unit test will solve the problem flawlessly.

The purpose of the tests to allow the implementations to be refactored / enhanced without too much worry about breaking the system. This seems to fail for your mock objects. There is a fundamental difference between say testing a encapsulated getter / setter pair in a class and the internal (non-public) order and set of OpenGL calls made in the implementation. If I change the getter, I can change the setter at the same time and the tests will still pass, regardless of what I did.

In summary, I think testing in this way is not a good idea and should be avoided. Sorry.

Roger

About comparing rendering results, well yes, it is not suitable for unit testing but can be a valuable tool in higher level testing mechanisms.

template
class render
{
public:
void doRendering()
{
OpenGLApi::glBegin( GL_POINT );
// vertex, texture, etc calls
OpenGLApi::glEnd();
}
};

For the real code, I have a class which just forward calls to the opengl functions, and in the unit tests I am checking the execution order (collecting them to the stringstream).

My point was that solution #1 – that you’ve actually referred as “naive”, “slow” and “expensive”, etc – is an important topic for several scientific reserches…

I was saying it is a “naive” solution for unit testing as it wouldn’t be real unit testing not talked about functional testing…

By the way now you keep saying that unit testing is for the early stage of development. Well, your OpenGL-based code mostly wont render anything in the early stage where you want to detect typing errors and such – in this case i don’t even see the point of mentioning solution #1.

Yes, you are right at this point. Mentioning solution #1 and actually the whole flow of finding a good solution is for the purpose of showing what wouldn’t be a suitable solution for unit testing and the reasons behind why we would like to take another step to find one. It is not for actual use, just illustrates the train of thought…

It is not easy to not take your posts as offense as we already heavily discussed about the topic with Chris (see above) so that checking for actual correct behavior on target hardware is a totally different story.

Testing code that does rendering without letting to do it’s job is like testing a car’s engine without even igniting it

I don’t agree with this as think about a complex class that invokes OpenGL function calls several times. How would you test if this class does exactly what it has to without having to put it together with the whole system? You have to mock out the OpenGL API anyway or you choose to setup a working OpenGL context and environment for your class but then if a problem pops up it is difficult to decide whether the problem is in the environment setup or in the actual class you would like to test. Also, this makes your code tightly coupled with the environment setup code and that can heavily affect the maintenance costs of the testing environment.

In case of image comparison you actually test the real and full functionality, but the execution needs dedicated hardware and quite a lot of time as comparing images is not like comparing two integers. Unit testing is to provide limited verification capabilities that provide fast regression executions thus providing smaller feedback cycles and identification of a subset of possible problems in the early phase of the development.

Comparing two things that serve different purposes is simply non-sense.