Simple Start

We’ll start by creating a C++ library. A very simple one; one function, throwing an exception derived from a generic library exception class. As follows:

// lib.cpp

#include "lib.h"
#include "exceptions.h"

void throwDerivedException()
{
  throw DerivedException();
}

Its header file is a one-liner:

// lib.h

extern void throwDerivedException();

And the exception definitions are inside a single header file:

// exceptions.h

class LibraryException: public std::exception {};
class DerivedException: public LibraryException {};

So that’s that. As I’m a big fan of testing, I’ll create a unit test for my newly created library. We’re focusing on a single problem here, so I’ll just create the necessary tests to trigger the problem I’m investigating. The problem is in the exception handling part, so I’ll create a simple test to make sure that exceptions thrown by throwDerivedException() can actually be caught. More specific, I want to check if my library generic LibraryException can be used to catch unforeseen errors inside my library functions. This obviously is trivial:

// test.cpp

#include "lib.h"
#include "exceptions.h"

int main(int argc, const char *argv[])
{
  // Exception handling check with statically compiled in symbols.
  std::cout << "Testing exceptions, statically linked: ";

  try {
    throwDerivedException();
  }
  catch (LibraryException &e)
  {
    std::cout << "ok\n";
  }
}

Compiling and running the above code will give results as expected:

$ g++ -c lib.cpp
$ g++ -c test.cpp
$ g++ -o test test.o lib.cpp
$ ./testTesting
exceptions, statically linked: ok

So far for C++ and statical linking, that leaves us still with a lot of remaining terms: Python, SWIG, dynamic linking, shared objects, position independent code, and linux loader. I’ll start using more of them very soon, so let’s continue.

Adding Complexity

A common way of supplying access to a library’s interface, besides its native (in our case C++) interface, is providing extensions for script languages. One of the best tools I’ve ever used to do that generically is SWIG. With SWIG you can generate wrapper code to create an interface that you can compile, together with your C++ code, as a shared library that can be dynamically loaded by a script language. In today’s example I’m going to connect my library interface from lib.h to Python. One particular interesting feature of SWIG is its language independent exception handling functionality. I’m going to use that to catch all exceptions that can be thrown by my library interface, identified by LibraryException.

I won’t go into all of SWIG’s details here, you can browse through its excellent online documentation that can be found on http://www.swig.org. I’ll just list the contents of my interface file here:

%module lib
%{
  #include "lib.h"
  #include "exceptions.h"
%}

/* Generic language independent exception handler. */
%include "exception.i"
%exception {
  try {
    $action
  }
  catch (LibraryException &e) {
    SWIG_exception(SWIG_RuntimeError, e.what());
  }
}

%include "lib.h"

Running SWIG on this interface file will generate a so called interface wrapper file, that contains the actual wrapper code to bind the C++ interface to a Python callable interface. In there the $action placeholder will be translated to a call to throwDerivedException() function for the respective Python wrapper function. In turn, a Python understandable RuntimeError will be raised.

All this is best illustrated with a simple Python test script to test the exception functionality, like we did for the C++ interface above:

# test.py
print "Testing exceptions, dynamically loaded with Python:",

# Import SWIG created library.
import lib

try:
  lib.throwDerivedException()
except RuntimeError:
  print "ok"

Compiling and running the above test gives results as expected:

$ swig -c++ -python -o lib_wrap.cpp lib.i
$ g++ -fPIC -c lib.cpp
$ g++ -fPIC -c -I/usr/include/python2.5 lib_wrap.cpp
$ g++ -fPIC -shared -o _lib.so lib_wrap.o lib.o
$ python test.py
Testing exceptions, dynamically loaded with Python: ok

Note that I now pass the -fPIC flag to gcc to emit position-independent code, suitable for dynamic linking.

So Where’s the Problem?

Everything fine so far, however, I’d like to combine both tests into one test executable, since running multiple tests manually is too cumbersome for me. Luckily there’s a function in the Python/C API called PyRun_SimpleFile to execute a Python script from within a C or C++ program. Including Python.h, linking with libpython, and calling Py_Initialize() is all there is to it. Let’s extend the C++ test program:

// test.cpp

#include "lib.h"
#include "exceptions.h"
#include "Python.h"

int main(int argc, const char *argv[])
{
  // Exception handling check with statically compiled in symbols.
  std::cout << "Testing exceptions, statically linked: ";

  try  {
    throwDerivedException();
  }
  catch (LibraryException &e) {
    std::cout << "ok\n";
  }

  // Embedded Python interpreter exception handling check -- this
  // will fail if test executable is statically linked.
  Py_Initialize();
  int result = PyRun_SimpleFile(fopen("test.py", "r"), "test.py");
}

For the test to run successfully, we need to add three lines before importing the generated library in the test script, otherwise it won’t be found:

...

# Add current dir to system path, that's where our lib is.
import sys
sys.path.append(".")

# Import SWIG created library.
import lib

...

So let’s compile this and run the test again:

$ swig -c++ -python -o lib_wrap.cpp lib.i
$ g++ -c -fPIC lib.cpp$ g++ -c -fPIC -I/usr/include/python2.5 test.cpp
$ g++ -c -fPIC -I/usr/include/python2.5 lib_wrap.cpp
$ g++ -fPIC -shared -o _lib.so lib_wrap.o lib.o
$ g++ -fPIC -o test test.o lib.o -lpython2.5
$ ./test
Testing exceptions, statically linked: ok
Testing exceptions, dynamically loaded with dlopen with Python:
terminate called after throwing an instance of 'DerivedException'
  what():  std::exception
Aborted

That’s not what we’d expected. The exception thrown inside the library is not caught by the exception handler code installed by SWIG. How could that happen?

The Problem Explained

After stripping down the actual problem into the simple test example as described, spending a few intense hours in a Googling-reading-Googling-reading loop, and interrogating a compiler guru colleague [1], I could conclude that I found an incarnation of the problem of RTTI crossing shared library boundaries. It’s actually a combination of how gcc does comparisons on the type of C++ objects and how symbols in dynamically shared libraries are resolved.

The entry “dynamic_cast, throw, typeid don’t work with shared libraries” in the GNU GCC FAQ explains the first half of the problem:

The new C++ ABI in the GCC 3.0 series uses address comparisons, rather than string compares, to determine type equality.

But also it describes the root cause of the other half of the problem:

Like other objects that have to be present in the final executable, these std::type_info objects have what is called vague linkage because they are not tightly bound to any one particular translation unit (object file). The compiler has to emit them in any translation unit that requires their presence, and then rely on the linking and loading process to make sure that only one of them is active in the final executable. With static linking all of these symbols are resolved at link time, but with dynamic linking, further resolution occurs at load time. You have to ensure that objects within a shared library are resolved against objects in the executable and other shared libraries.

So what happens in our example? First, the executable is read by the loader, including the exception symbols LibraryException and DerivedException. The C++ interface test is executed, giving the expected results. The Python test will be run, with the embedded Python interpreter. While interpreting the Python test script, the _lib.so library will be loaded. At load time of the library the types inside catch statements are resolved using the symbols in memory of the test executable. In the execution path the throwDerivedException() function will be called. At that moment, at run time, the exception instance types will be resolved, using the local symbols of the library. The exception object that is thrown will have a pointer pointing to the address of the LibraryException type inside the library memory space, while the catch statement inside the wrapper code will check using the address of the LibraryException type from inside the test executable memory space: they won’t match. Hence the exception won’t be caught; program execution aborts.

If you don’t believe me, you can see for yourself. Here’s a link to an archive containing the sample code as discussed. It contains a Makefile too, so just type:

$ make
$ ./test

And there you go. By means of the included Makefile, the archive also contains the three different solutions which I’ll discuss next.

It’s All in the Options

Sometimes a problem that seems to be very complicated in the first place — taking a significant investigation to get to the bottom of it — can have a relatively simple solution. In this particular situation we even have three trivial solutions. As if we have won the lottery.

What we need to do is somehow to instruct the compiler, the linker, or the loader to use the right symbols to resolve the exception types. In other words, we need to make sure that resolving the exception types is not being messed up by either the linker or the loader. We can accomplish that by:

1. Using shared libraries — if we compile lib.cpp as a shared library say liblib.so, the test executable will not contain exception type symbols. The exception symbols inside liblib.so will remain local and while _lib.so is loaded by Python the exception symbols in there will be resolved locally too.
2. Exporting symbols to the dynamic symbol table — by exporting all symbols from the test executable to the dynamic symbol table, not only the type of the exception in the catch statement, but also the type of the exception in the throw statement will be resolved inside the executable memory space. The GNU linker has a special option for this purpose: -E, which, according to the GNU ld manual, “when creating a dynamically linked executable, add[s] all symbols to the dynamic symbol table. The dynamic symbol table is the set of symbols which are visible from dynamic objects at run time. If you do not use this option, the dynamic symbol table will normally contain only those symbols which are referenced by some dynamic object mentioned in the link. If you use dlopen to load a dynamic object which needs to refer back to the symbols defined by the program, rather than some other dynamic object, then you will probably need to use this option when linking the program itself.”
3. Binding references to global symbols to the definition within the shared library — using the -Bsymbolic option from the GNU linker. If we link the _lib.so Python extension with this option, the exception type inside the catch statement will be resolved already to the local symbols inside the library by the linker at link time. When the library is loaded dynamically by Python in the test run, no symbols will be overridden with symbols from the test executable. I chose this third solution as my solution of choice, because it solves the problem inside the Python extension library. As a result, the Python extension becomes fully self contained, which in the particular case I was working on was exactly what I wanted.

These solutions can be tried out using the archive as linked before. The Makefile has, besides the test target, three additional targets:

$ make test_shared
$ make test_export
$ make test_symbolic

that build a differently linked test executable, according to the above described solutions.

Enough compiler-and-linker-fun for me for this week…