Binary Compatibility Issues With C++

Definition

A library is binary compatible, if a program linked dynamically to a former version of the library continues running with newer versions of the library without the need to recompile.

If a program needs to be recompiled to run with a new version of library but doesn't require any further modifications, the library is source compatible.

Binary compatibility saves a lot of trouble. It makes it much easier to distribute software for a certain platform. Without ensuring binary compatibility between releases, people will be forced to provide statically linked binaries. Static binaries are bad because they

In the KDE project, we will provide binary compatibility within the life-span of a major release.

The do's and don'ts

You can ...

You cannot ...

Techniques For Library Programmers

The biggest problem when writing libraries is, that one cannot safely add data members since this would change the size and layout of every class, struct, or array containing objects of the type, including subclasses.

Bitflags

One exception are bitflags. If you use bitflags for enums or bools, you can safely round up to at least the next byte minus 1. A class with members 

	uint m1 : 1;
	uint m2 : 3;
	uint m3 : 1;
can safely be extended to 

	uint m1 : 1;
	uint m2 : 3;
	uint m3 : 1;
	uint m4 : 2; // new member
without breaking binary compatibility. Please round up to a maxmimum of 7 bits ( or 15 if the bitfield was already larger than 8). Using the very last bit may cause problems on some compilers.

Using a d-Pointer

Bitflags and predefined reserved variables are nice, but far from being sufficient. This is where the d-pointer technique comes into play. The name "d-pointer" stems from Trolltech's Arnt Gulbrandsen, who first introduced the technique into Qt, making it one of the first C++ GUI libraries to maintain binary compatibility even between bigger release. The technique was quickly adapted as general programming pattern for the KDE libraries by everyone who saw it. It's a great trick to be able to add new private data members to a class without breaking binary compatibiliy.

Remark: The d-pointer pattern has been described many times in computer science history under various names, e.g. as pimpl, as handle/body or as cheshire cat. Google helps finding online papers for any of these, just add C++ to the search terms.

In your class definition for class Foo, define a forward declaration 

class FooPrivate;
and the d-pointer in the private section: 

private:
	FooPrivate* d;
The FooPrivate class itself is purely defined in the clas implementation file (usually *.cpp ), for example: 

class FooPrivate {
public:
	FooPrivate()
	: m1(0), m2(0)
	{};
	int m1;
	int m2;
	QString s;
};
All you have to do now is to create the private data in your constructors or your init function with 

	d = new FooPrivate;
and to delete it again in your destructor with 

	delete d;

You may not want all member variables to live in the private data object, though. For very often used members, it's faster to put them directly in the class, since inline functions cannot access the d-pointer data. Also note that all data covered by the d-pointer is obviously private. For public or protected access, provide both a set and a get function. Example 

	QString Foo::string() const
	{
		return d->s;
	}

	void setString( const QString& s )
	{
		d->s = s;
	}

Trouble shooting

If you don't have free bitflags, reserved variables and no d-pointer either, but you absolutely have to add a new private member variable, there are still some possibilities left. If your class inherits QObject, you can for example place the additional data in a special child and find it by traversing over the list of children. You can access the list of children with QObject::children(). However, a fancier and usually faster approach is to use a hashtable to store a mapping between your object and the extra data. For this purpose, Qt provides a pointer-based dictionary called QPtrDict.

The basic trick in your class implementation of class Foo is:

  1. Create a private data class FooPrivate.
  2. Create a static QPtrDict<FooPrivate>.

    Note that some compilers/linkers (almost all, unfortunately) do not manage to create static objects in shared libraries. They simply forget to call the constructor. Therefore you need a static pointer to a QPtrDict, and an access function: 

    	// BCI: Add a real d-pointer
	static QPtrDict<FooPrivate>* d_ptr = 0;
	static void cleanup_d_ptr()
	{
		delete d_ptr;
	}
	static FooPrivate* d( const Foo* foo )
	{
		if ( !d_ptr ) {
			d_ptr = new QPtrDict<FooPrivate>
			qAddPostRoutine( cleanup_d_ptr );
		}
		FooPrivate* ret = d_ptr->find( (void*) foo );
		if ( ! ret ) {
			ret = new FooPrivate;
			d_ptr->replace( (void*) foo, ret );
		}
		return ret;
	}
	static void delete_d( const Foo* foo )
	{
		if ( d_ptr )
			d_ptr->remove( (void*) foo );
	}

  3. Now you can use the d-pointer in your class almost as simple as in the code before, just with a function call to d(this). For example: 

    	d(this)->m1 = 5;

  4. Add a line to your destructor: 

    	delete_d(this);
    That's not really required, but saves some resources.

  5. Do not forget to add a BCI remark, so that the hack can be removed in the next version of the library.
  6. Do not forget to add a d-pointer to your next class.

Matthias Ettrich [email protected]