25
Mar 12 Java DOM’s Iteration Irritation
Posted by Trevor Pounds under , , ,
2 comments

The last time I had to interact with Java’s DOM API I noticed that it had not been updated to make use of any recent Java 5 enhancements. Specifically, the org.w3c.dom.NodeList interface does not implement java.lang.Iterable. The unfortunate side effect is ugly and error prone code.

// iteration sadness :(
final NodeList nodes = document.getChildNodes();
for(int i = 0; i < nodes.getLength(); i++) {
  final Node node = nodes.item(i);
  // use node
}

Ideally, the code could have been written utilizing Java 5′s foreach loop.

// iteration happiness which is currently impossible
for(final Node node : document.getChildNodes()) {
  // use node
}

As a workaround, I’ve written a simple wrapper class for org.w3c.dom.NodeList that makes it iterable. Hopefully, it’s useful for others.

Download: IterableNodeList.java

package com.trevorpounds.dom.util;
 
import java.util.Iterator;
import java.util.NoSuchElementException;
 
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
 
/**
 * Simple org.w3c.dom.NodeList iterable wrapper
 * that can be used with Java's foreach loop.
 *
 * @author Trevor Pounds
 */
public class IterableNodeList implements Iterable<Node>, Iterator<Node> {
   /** List of nodes */
   private final NodeList nodeList;
 
   /** List of nodes current index. */
   private int index = 0;
 
   /** Constructs iterable node list. */
   public IterableNodeList(final NodeList nodeList) {
      this.nodeList = nodeList;
   }
 
   /** {@inheritDoc} */
   public boolean hasNext() {
      return index < nodeList.getLength();
   }
 
   /** {@inheritDoc} */
   public Iterator<Node> iterator() {
      return this;
   }
 
   /** {@inheritDoc} */
   public Node next() throws NoSuchElementException {
      if(!hasNext()) {
         throw new NoSuchElementException();
      }
      return nodeList.item(index++);
   }
 
   /** {@inheritDoc} */
   public void remove() throws IllegalStateException, UnsupportedOperationException {
      throw new UnsupportedOperationException();
   }
}

Utilizing the wrapper class we can rewrite the sad code above to the following.

for(final Node node : new IterableNodeList(document.getChildNodes())) {
  // use node
}

28
Mar 11 JNI: Emulating Pointer-to-Pointers
Posted by Trevor Pounds under , ,
4 comments

I recently ran into an instance where I wanted to use several native libraries directly in Java and have the method signature model the original C API as closely as possible, allowing me to store opaque pointer types directly within a Java class. For anyone not familiar, opaque pointers are typically stored within Java classes using the long primitive type which is capable of storing most target platforms native pointer types (i.e. 8 or less bytes). One drawback of storing native pointers in pure Java is a lot of C API calls allocate resources using pointer-to-pointers or double pointers (i.e. sqlite3_open(…)). Unfortunately, JNI does not have a concept for manipulating pointer-to-pointers or reference types. That said, it is possible to emulate a pointer-to-pointer via JNI by using a primitive array wrapper (i.e. long[]).

For example suppose I wanted to model the following C function that returns a simple pass/fail result and outputs a newly allocated integer using a pointer-to-pointer.

int alloc_opaque_type(T** ppT);

The JNI equivalent would instead substitute the method’s first argument with a long primitive array.

public static native int alloc_opaque_type(long[] ppT);

The native JNI implementation then requires a slight pointer fixup between the Java primitive array type and the native pointer type which is accomplished using a simple union cast below.

JNIEXPORT jint JNICALL Java_Wrapper_alloc_1opaque_type
   (JNIEnv* env, jclass klass, jlongArray ppT)
{
   union
   {
      T* p;
      jlong    l;
   } __u;
   int retval = alloc_opaque_type(&__u.p);
   env->SetLongArrayRegion(ppT, 0, 1, &__u.l);
   return retval;
}

After the native code is compiled and loaded, you can then use the method with an opaque pointer as follows:

public long allocAndGetPointer()
{
   long[] opaque_ptr_ptr = new long[] { 0 };
   if(alloc_opaque_type(opaque_ptr_ptr))
      { return long[0]; }
   throw new RuntimeException("Could not allocate opaque pointer!");
}

27
Sep 10 Java Internals: Unavoidable Object Overhead
Posted by Trevor Pounds under ,
2 comments

A few months back I completed work on an internal library at AMD to optimize memory usage for reading a large number (tens of millions) of unique records into memory from a compressed proprietary data format. Overall the process contained many new and interesting challenges, most related to reducing the current bloated data representations down to something manageable. I won’t get too much into the details but it boiled down to several strategies: substitution of primitive wrappers (i.e. java.lang.Integer) with the corresponding primitive formats, real-time bytecode generation and object interning. Although, these strategies offered impressive memory compression (> 90% in some cases!) over earlier version of the library, one problem still remained, the unavoidable overhead of java.lang.Object’s internal header. Unfortunately, the java.lang.Object header overhead is incurred by all Java objects and is tragically unavoidable. Typically, Java VM’s make use of an internal VM object model header consisting of at least two fields: a klass_type pointer and a lock_type pointer. Additionally, a Java Array object includes an additional field for the length.

class VMObject
{
   klass_type* klass;
   lock_type*  lock;
};

Typical Internal VM Object Representation
 
class VMArray : VMObject
{
   size_t length;
};

Typical Internal VM Array Representation

I won’t get into too much detail about the klass_type pointer since this one speaks for itself. It’s primary purpose is to provide all of the runtime type and method dispatch information to the VM. The lock_type is also straight forward providing the internal mechanism for Java’s synchronized keyword. However, it also happens to double every object’s overhead whether or not synchronization is used at all. A good resource that I found touching on the subject is available on the c2.com wiki which outlines the pitfalls of the Java object model and parallels my own independent discoveries. Unfortunately, Java was never designed for data-only types (i.e. structs in C/C++). As a result, all objects in Java require both fields regardless of synchronization or virtual dispatch. I came to the same conclusion as the authors at c2.com wiki; the memory overhead can only be reduced in two ways: (1) provide a method to mark objects as unsynchronizable and (2) incorporate a data-only type. The changes are not impossible; in fact both could be retrofitted into future Java releases with a little bit of effort. The addition of an java.lang.concurrent.Unsynchronizable interface would provide a backward compatible way for the Java compiler and VM’s object model to support a lockless subtype. Supporting a data-only type is a bit trickier but could be achieved again using an updated Java object model in addition to some compile-time and run-time optimizations.


19
Feb 10 Using Multiple Java Annotations
Posted by Trevor Pounds under , , ,
0 comments

The other day I was experimenting with some concepts for JUnitBench related to user specified metadata supplied via annotation types. The concept would allow a developer to supply additional column data to the performance measurement output, specifically CSV, which can be used in custom post-processing calculations. The basic interface I wanted to provide for the metadata was something along the lines of the following:

@Metadata(name="size (KB)", value="1024")
@Metadata(name="size (MB)", value="1")
@Test public void myPerfTest()
{
   // do stuff
}

Unfortunately, multiple annotations are not allowed; this is apparent when you try to compile via javac.

./org/junitbench/examples/MetadataTest.java:[23, 18] duplicate annotation

Additionally, the java reflection API does not provide a mechanism for retrieving multiple annotations by name (see: java.lang.reflect.AnnotatedElement.getAnnotation(…)). If it was explicitly allowed by the language I could probably scrape by using one of the existing reflection API’s get*Annotations(…) methods with a custom annotation filter. However, since it is not allowed in any capacity another workaround was required; I did this by defining an enclosing annotation type that accepted an unlimited number of nested annotation types.

@Metadata({
   @Data(name="size (KB)", value="1024"),
   @Data(name="size (MB)", value="1")
})
@Test public void myPerfTest()
{
   // do stuff
}

The above annotation types are defined as follows:

@Inherited
@Retention(RetentionPolicy.RUNTIME)
@Target({})
public @interface Data
{
   String name();
   String value();
}
@Inherited
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.METHOD})
public @interface Metadata
{
   Data[] value();
}

The important thing to be aware of is the use of the nested @Data annotation as the array value type within the enclosing @Metadata annotation definition.


15
Feb 10 Detecting GCC Versions with SCons
Posted by Trevor Pounds under , , , , , , ,
0 comments

A few weeks ago I was tasked to add support for a C++ project’s build system to detect unknown (i.e. currently unsupported) platforms and compilers. At the time, the project only supported platforms for which we had built pre-compiled binaries, including several GNU/Linux/GCC, Windows/MSVC, and Solaris/GCC targets. The build system is based on SCons, which by extension supports all of the functionality available from the Python language. As a result, I was able to utilize portable Python functionality to supplement the current build system’s platform/compiler detection scheme. The first step of the detection algorithm involved parsing the results from Python’s os.uname() and setting several build system variables. Then, the compiler environment is configured by pulling the CXX SCons environment variable, matching it against supported types (e.g. GCC, MSVC, etc.) and performing a compiler dependent version look-up. A fairly simple approach used to determine the GCC version can be done by adding similar code to a SCons build script.

env = Environment()
GCC_VERSION = commands.getoutput(env['CXX'] + ' -dumpversion')

Note, in the example I opted to use the -dumpversion flag instead of the -v or –version flags because from my experience I have found the results can be inconsistent, from a parsing aspect, depending on the GCC build being used (i.e. differences in vanilla and GNU/Linux distribution patched builds). After detecting the version, the results are used to configure the appropriate compile time and link flags (e.g. -O1 vs. -O2, -m32 vs. -m64, etc.). The version is also used to determine which run-time dependencies (i.e. libstdc++.so, libgcc_s.so) need to be linked and distributed with the application. Fortunately, a convenient mechanism exists, the -print-file-name option, which can be used to determine GCC’s library awareness. For example, the option can be used to resolve the libstdc++.so and libgcc+_s.so dependencies as follows.

g++-4.1 -print-file-name=libgcc_s.so.1
/usr/lib/gcc/x86_64-linux-gnu/4.1.3/../../../../lib/libgcc_s.so.1

g++-4.1 -m32 -print-file-name=libgcc_s.so.1
/usr/lib/gcc/x86_64-linux-gnu/4.1.3/../../../../lib32/libgcc_s.so.1

g++-4.1 -print-file-name=libstdc++.so.6
/usr/lib/gcc/x86_64-linux-gnu/4.1.3/../../../../lib/libstdc++.so.6

g++-4.1 -m32 -print-file-name=libstdc++.so.6
/usr/lib/gcc/x86_64-linux-gnu/4.1.3/../../../../lib32/libstdc++.so.6

Additionally, the -print-file-name option works in conjunction with the -m32 or -m64 flags. This allows the build system, depending on the bit-ness of the target system, to resolve the absolute path of the respective i386 or x86-64 compatible libraries. However, I did find that the -print-file-name option in GCC versions <= 3.2.1 seemed to resolve to invalid paths in some circumstances. For those particular compilers another mechanism would need to be used.

As an addendum, I've compiled a quick cross-reference of various GCC features which I acquired from some of my previous work as well as utilizing some of the compiler options above.

GCC libstdc++ multilib (i386/x86-64) -dumpversion -print-file-name
2.95.x 3? no works not tested
3.0.x 3 no works not tested
3.1.x 4 yes * works not tested
3.2.x – 3.3.x 5 yes * works works **
3.4.x – 4.4.x 6 yes works works

* Technically i386/x86-64 multilib support is available in these versions but does not seem to be as widley supported. See the notes in my previous post for more info.

** Does not work in all cases. In my limited amount of testing I found that GCC 3.2.1 and earlier had trouble resolving libstdc++.so shipped with the compiler. I would test this option thoroughly if it needs to be used with older compilers.


23
Jan 10 Building Older GCC Versions on x86-64 Debian/Ubuntu
Posted by Trevor Pounds under , , , , , , , , ,
36 comments

Some time ago, while working on MockitoPP, I was interested to see what compilers were compatible with several non-standard compiler dependent techniques (e.g. vtable, data/code pointer sizes) in use. Early on I focused most of my attention on supporting the newer, more popular GCC (i.e. 4.x) and Visual Studio (e.g. VS .Net 2003) compilers. After I hardened the code base a bit I decided to see how well it faired with older versions of GCC. For the purposes of my experiment I decided to test against the latest GCC release from each major series (i.e. 2.95.x to 4.4.x). The build and target system I used for testing was Ubuntu 9.10 on an x86-64 architecture. The main features I wanted were the standard C/C++ languages and, if available, multilib (e.g. i386/x86-64) support. Unfortunately, building an older compiler using a newer tool-chain is not a trivial task in many circumstances; normally a new compiler is compiled using an older tool-chain! One obvious problem is that GCC compiler features have changed over the years which has complicated over time with the advent of more diverse Linux distos. It gets even worse when trying to build a usable 32-bit host/target compiler on a system with a 64-bit tool-chain (see: GCC 2.95.x and 3.0.x below). Additionally, older versions of GCC have limited multilb awareness support on Linux distributions that do not use a “standard” directory structure. For example, in most LSB 64-bit distributions the following lib path convention is used:

/lib – 32-bit libraries
/lib64 – 64-bit libraries

However, on Debian based systems the following convention is used:

/lib – 64-bit libraries
/lib64 – symlink to /lib
/lib32 – 32-bit libraries

Unfortunately, this convention confuses the older GCC configuration scripts. However, this limitation has been resolved in later releases.

GCC >= 4.2.4
To start off, I’ll talk about building modern GCC versions. From my limited testing I found that any version starting with 4.2.4 and later can be built by following the standard GCC build procedure outlined on the web. However, for documentation purposes, I’ve listed the commands I used to build and install GCC 4.4.2 to a custom installation prefix (e.g. /opt).

tar xzf gcc-4.4.2.tar.gz
mkdir gcc-4.4.2-objdir
cd gcc-4.4.2-objdir
../gcc-4.4.2/configure --prefix=/opt/x86_64/gcc/gcc-4.4.2 --enable-languages=c,c++
make
make install

multilib support is auto-enabled using the above configure options but it can also be explicitly enabled by adding –enable-multilib to the argument list.

GCC 3.3.x to 4.2.3
Building versions 3.3.x to 4.2.3 is a little more involved but essentially boils down to applying a slightly modified version of the lib path awareness patch mentioned above. For simplicity’s sake I’ve attached the patch (gcc-multilib-fix-v3.3.x-to-v4.2.3.debian.x86_64.diff) and used the following commands to build the compiler.

tar xzf gcc-3.4.6.tar.gz
cd gcc-3.4.6
patch -p0 < gcc-multilib-fix-v3.3.x-to-v4.2.3.debian.x86_64.diff
mkdir ../gcc-3.4.6-objdir
cd ../gcc-3.4.6-objdir
../gcc-3.4.6/configure --prefix=/opt/x86_64/gcc/gcc-3.4.6 --enable-languages=c,c++
make
make install

GCC 3.2.x
Building GCC 3.2.x requires following the same basic steps as above but requires an additional fix that prevents emitting an .eh_frame for the crt*S.o objects. The original bug fix can be found on the GCC mailing list. To build this version of GCC, download the attached patch (gcc-multilib-fix-v3.2.x.debian.x86_64.diff) and perform the following commands.

tar xzf gcc-3.2.3.tar.gz
cd gcc-3.2.3
patch -p0 < gcc-multilib-fix-v3.2.x.debian.x86_64.diff
mkdir ../gcc-3.2.3-objdir
cd ../gcc-3.2.3-objdir
../gcc-3.2.3/configure --prefix=/opt/x86_64/gcc/gcc-3.2.3 --enable-languages=c,c++
make
make install

multilib i386/x86-64 support is available in this version, however, I seem to recall reading on the web that proper x86-64 support did not become production ready until the 3.3.x releases. At the time of this post I could not find any resource to back this claim. However, I did find the quote, "The x86-64 port has been significantly improved," in the 3.3.x change log which potentially confirms this. In any case, I compiled a few simple programs for an x86-64 target and none of them had any run-time problems as far as invalid calculations or heinously crashing but if anyone knows for sure I would be interested to know.

GCC 3.1.x
Up until now most of the builds were fairly easy to "frontport" to run on x86-64 Debian. With GCC 3.1.x, I began to run into more difficult problems. Along with the .eh_frame patch used above, this version needed work to fix the lib32/lib64 path lookup mechanisms for multilib support; FWIW, this is the first GCC version that supported x86-64 host/targets. Also, an additional workaround was needed to forward some older libc symbols (e.g. __ctype_*) to their corresponding available equivalents (e.g. __ctype_*_loc). I've compiled the corresponding fixes into a patch (gcc-multilib-fix-v3.1.x.debian.x86_64.diff) which can be used to build this version.

tar xzf gcc-3.1.1.tar.gz
cd gcc-3.1.1
patch -p0 < gcc-multilib-fix-v3.1.x.debian.x86_64.diff
mkdir ../gcc-3.1.1-objdir
cd ../gcc-3.1.1-objdir
../gcc-3.1.1/configure --prefix=/opt/x86_64/gcc/gcc-3.1.1 --enable-languages=c,c++ --enable-threads=posix
make
make install

Without the --enable-threads=posix flag, the configure script will build using the default thread model (e.g single). If you do not wish to support posix threads the option can be omitted.

GCC 3.0.x
As I mentioned above, the GCC 3.1.x series of compilers were the first to natively support x86-64 targets. In order to build GCC 3.0.x (for an i386 target) on a x86-64 host the source required a change to the GCC standard library prefix directories (e.g. /lib to /lib32) and a few tool-chain workarounds (e.g. -m32 flags). Probably, the most interesting part of the changeset is the addition of --32 to the GCC assembler spec options. Without the flag, the assembler chokes with an invalid operand error (e.g. Error: suffix or operands invalid for `push'). Alternatively, I could have specified -Wa,--32 on the GCC command line to workaround the tool-chain differences but doing this is more burdensome and error prone than having the default behavior compiled into the spec file. Additionally, the same __ctype_* symbol lookup change in the GCC 3.1.x patch is also required. Again, I've provided the compatibility changes in a patch (gcc-v3.0.x.debian.x86_64.diff) which can be built using the following commands.

tar xzf gcc-3.0.4.tar.gz
cd gcc-3.0.4
patch -p0 < gcc-v3.0.x.debian.x86_64.diff
mkdir ../gcc-3.0.4-objdir
cd ../gcc-3.0.4-objdir
../gcc-3.0.4/configure --prefix=/opt/i386/gcc/gcc-3.0.4 --enable-languages=c,c++ --enable-threads=posix --host i386-pc-linux-gnu
make
make install

GCC 2.95.x
Finally! We come to the GCC 2.95.x series which, rightfully so, required quite a bit of effort to configure and build for an x86_64 host. About half of the changes needed for GCC 3.0.x are also applicable here (e.g. Makefile fixes, lib32 fix-ups, spec flags, etc). Unfortunately, the other half are workarounds for compiler and glibc nuances that are fixed in later releases. One of the big changes needed is a workaround for the internal use of an _IO_MTSAFE_IO define. This define causes an error due to a missing header due to difference in glibc headers included with modern Linux distributions (see here and here for more info). Luckily, a straightforward workaround is to use glibc's generic stdio-lock.h header, which in my case needed to be pulled from the glibc 2.10.1 source to match my installed version. Additionally, another glibc fix is required due to an unnamed union declaration (see this bug) inside a pthread header which is not supported by GCC 2.95.x. To workaround the issue I pulled in the glibc 2.10.1 system dependent pthreadtypes.h header from the source and appended a variable name (e.g. __gcc_295_workaround__) to the unnamed union declaration. Build this version using the provided patch (gcc-v2.95.x.debian.x86_64.diff) and the following instructions.

tar xzf glibc-2.10.1.tar.gz
mkdir -p gcc-2.95.3/glibc-workaround/include/bits
cp glibc-2.10.1/bits/stdio-lock.h gcc-2.95.3/glibc-workaround/include/bits
cp glibc-2.10.1/nptl/sysdeps/unix/sysv/linux/x86_64/bits/pthreadtypes.h gcc-2.95.3/glibc-workaround/include/bits
sed -i -n '1h;1!H;${;g;s/\(__pthread_slist_t __list;\n[ \t]*}\)/\1 __gcc_295_workaround__/g;p;}' gcc-2.95.3/glibc-workaround/include/bits/pthreadtypes.h
tar xzf gcc-2.95.3.tar.gz
cd gcc-2.95.3
patch -p0 < gcc-v2.95.x.debian.x86_64.diff
mkdir ../gcc-2.95.3-objdir
cd ../gcc-2.95.3-objdir
../gcc-2.95.3/configure --prefix=/opt/i386/gcc/gcc-2.95.3 --enable-languages=c,c++ --enable-threads=posix --enable-shared --host i386-pc-linux-gnu
make
make install

Be sure to use the same version of the glibc source for the patches as the one of the target system. Since my target/host system uses glibc 2.10.1 I made sure to pull the corresponding source.

Additional Notes
I used GCC 3.4.6 in all cases when building the GCC 2.95.x and 3.x.x versions. The host compiler can be configured by setting the CC environment variable to the full path of your compiler (e.g. CC=/opt/x86_64/gcc/gcc-3.4.6/bin/gcc). All of the GCC 4.x.x versions can pretty much be built with any of the older GCC compilers. However, your mileage may vary in some circumstances. In those cases you can always try to build with a different host compiler when the current one is not working out. What seemed to work best for me was building each subsequent version in descending order and bootstrapping a failed build using an older GCC host if necessary.


19
Oct 09 Linking to Older Versioned Symbols (glibc)
Posted by Trevor Pounds under , , , , , , ,
6 comments

In my last post I gave a brief overview of the mechanism used internally by glibc for versioning symbols within shared libraries. As an addendum to my previous article I would like to discuss a simple way to force linking against older glibc symbols. Why would you do this you may ask? Well, suppose you have several GNU/Linux systems with varying glibc installs across them but you want to deliver a binary that would be compatible across them. One option is to statically link your binary, my vote :) , the other is to link to an older symbol within the shared library. I would like to mention that using an older symbol has the obvious drawback, possible advantage, of using something that was deprecated for a reason in the past (i.e. broken, behavior changes, performed poorly, new arch support, etc.). The reason I say it is possibly advantageous to link against an older symbol may be because it is known to behave in a desired way, broken or not! I present the following merely for education purposes, so use with care.

To force linking against a particular symbol you need to use the same .symver pseudo-op that is used for defining versioned symbols in the first place. In the following example I make use of glibc’s realpath, but want to make sure it is linked against an older 2.2.5 version.

#include <limits.h>
#include <stdlib.h>
#include <stdio.h>
 
__asm__(".symver realpath,realpath@GLIBC_2.2.5");
int main()
{
   char* unresolved = "/lib64";
   char  resolved[PATH_MAX+1];
 
   if(!realpath(unresolved, resolved))
      { return 1; }
 
   printf("%s\n", resolved);
 
   return 0;
}

If you were to use objdump on the resulting binary you would see that it is indeed using realpath@GLIBC_2.2.5! Also note that other symbols have been resolved to their defaults so you need to make sure you add a .symver pseudo-op for each symbol you want to force to an older version.

0000000000000000      F *UND*  0000000000000000         realpath@GLIBC_2.2.5
...
0000000000000000      F *UND*  0000000000000000         __stack_chk_fail@@GLIBC_2.4

The .symver pseudo-op can be used this way to force any symbol to be linked against an older one so long as it is valid. To ease linking against older glibc versions I’ve provided a simple header which can be used to force linking against the minimum glibc version for a give x86 architecture. The minimum versions I am using were taken from shlib-versions file of the glibc git tree.

#ifndef __GLIBC_COMPAT_SYMBOL_H__
#define __GLIBC_COMPAT_SYMBOL_H__ 1
 
/**
 * add other architecures below
 */
#ifdef __amd64__
   #define GLIBC_COMPAT_SYMBOL(FFF) __asm__(".symver " #FFF "," #FFF "@GLIBC_2.2.5");
#else
   #define GLIBC_COMPAT_SYMBOL(FFF) __asm__(".symver " #FFF "," #FFF "@GLIBC_2.0");
#endif /*__amd64__*/
 
#endif /*__GLIBC_COMPAT_SYMBOL_H__*/

To use the glibc compatible header with the realpath example above, we merely use the GLIBC_COMPAT_SYMBOL macro with the appropriate symbol:

GLIBC_COMPAT_SYMBOL(realpath)

12
Oct 09 Versioning Symbols for Shared Libraries (glibc)
Posted by Trevor Pounds under , , , , , , ,
0 comments

Sometime back I got a request for running a piece of Java software on an older development system (2.4 kernel/2.2 glibc). The software makes use of a nifty little native compiled JVM launcher, Java Service Wrapper, which provides several features related to JVM configuration and life-cycle process management. Unfortunately, this software has a requirement on >= glibc 2.3 which isn’t supported by that system. Initially this did not surprise me, considering how old it was, but it did spark my curiosity as to how the dynamic loader enforces prerequisite versions on symbols even if an older version exists. Clearly shared library compatibility doesn’t end at versioning of the soname.

To get a basic idea of what shared libraries and symbols were listed in the wrapper binary I ran the following command:

objdump -x wrapper-linux-x86-32

Which gives quite a bit of useful info about the executable:

...
Dynamic Section:
NEEDED               libm.so.6
NEEDED               libpthread.so.0
NEEDED               libc.so.6
...
Version References:
required from libpthread.so.0:
0x0d696912 0x00 05 GLIBC_2.2
0x0d696911 0x00 04 GLIBC_2.1
0x0d696910 0x00 03 GLIBC_2.0
required from libc.so.6:
0x0d696913 0x00 08 GLIBC_2.3
0x0d696911 0x00 07 GLIBC_2.1
0x0d696912 0x00 06 GLIBC_2.2
0x0d696910 0x00 02 GLIBC_2.0
...
SYMBOL TABLE:
...
00000000    F *UND*  00000167      strchr@@GLIBC_2.0
00000000    F *UND*  00000078      nanosleep@@GLIBC_2.0
...
00000000    F *UND*  00000496      realpath@@GLIBC_2.3
...

In the output above I’ve omitted the uninteresting parts and left the important tidbits for the purposes of this article. (1) We can see that this binary requires the shared libraries: libm.so.6, libpthread.so.0, libc.so.6, (2) it has various glibc version references: 2.0, 2.1, 2.2, 2.3, and (3) has some interesting symbols appended with a unique version suffix.

So what? Well, after grepping through the large output from objdump I found that the only requirement on glibc 2.3 was the realpath function. Hmm…that’s interesting, if it wasn’t for this one function the executable would be binary compatible with the glibc installed on this old 2.4 linux distribution. Also, after browsing through the git tree I found that realpath has been available since at least glibc 2.1 (i.e. stdlib/canonicalize.c@202). So why is the dynamic linker requiring a newer version of glibc even though the function has been around for so long? The answer has to do with the versioned symbol scheme introduced in glibc 2.1 (http://people.redhat.com/drepper/symbol-versioning) which is an “extension” of Sun’s own symbol versioning scheme (http://docs.sun.com/app/docs/doc/817-1984/appendixb-45356?a=view).

As I mentioned above realpath has been around since glibc 2.1, yet this executable requires 2.3. The mechanism behind this is that the linker can be used to create global versioned symbol aliases to local symbols generated at compiled time. The internal linker interface provides two mechanisms for defining the aliases and requires the source to be inlined with a simple assembly pseduo-op .symver.

.symver actual, alias@version

.symver actual, alias@@version

The single @ op can be used to define any number of versioned symbols with the same base name. The double @@ can only be defined once for a given symbol since it denotes the default version to use. The linker also requires the use of a map file for defining if a symbol is global or local. A global symbol can be exported, while a local symbol is kept private. Each entry within the map file should correspond to a given version within the source file’s .symver definition.

VER_1.0 {
   global: alias
   local: *
};

To illustrate the full mechanics of the versioned symbol linking mechanism I’ve provided a better example. Say for example we release a library version 1.0 with a function foo. We could version it by adding the simple assembly to the function definition in the source foo.c:

__asm__(".symver foo, foo@@FOO_1.0");
int foo() {
   return 0;
}

Then we would need to create a map file (foo.map):

FOO_1.0 {
   foo;
};

After which we would compile our simple shared library.

gcc -shared -fPIC -Wl,--version-script foo.map foo.c -o libfoo.so.1

At this point we could distribute the library with the foo.h and let others compile against it to their hearts content. But what happens when we want to release another version of the library with potential behavioral changes that may affect the use of foo in other programs? We could introduce a new foo with with the updated behavior and still keep the old fucntion around for legacy programs. This can be done by renaming the original definition of foo to foo_1_0 and adding a new definition of foo called foo_1_1, like so:

/* old foo */
__asm__(".symver foo_1_0, foo@FOO_1.0");
int foo_1_0() {
   return 0;
}
 
/* new foo */
__asm__(".symver foo_1_1, foo@@FOO_1.1");
int foo_1_1() {
   return -1;
}

Then we update our existing map file (foo.map) to the following:

FOO_1.0 {
   foo;
};
FOO_1.1 {
   foo;
} FOO_1.0;

After which we would compile our simple shared library again and redistribute to our customers noting the change in behavior. Programs compiled against the old version and new version can now operate concurrently utilizing the updated library.