Loading Multiple Files - same name, different directories

Rod Evans — Sunday May 08, 2005

A recent customer observation reminded me of a subtlety of shared object dependency lookup, and a change that occurred between Solaris 8 and 9. The customer observed different dependencies being loaded on the two systems, although the applications file system hierarchy was the same on the two systems.

On Solaris 8, the following was observed.

  $ ldd ./app
      libX.so.1 =>     /opt/ISV/weblib/libX.so.1
      libY.so.1 =>     /opt/ISV/weblib/libY.so.1
      libZ.so.1 =>     /opt/ISV/weblib/libZ.so.1

And, on Solaris 9, the following was observed.

  $ ldd ./app
      libX.so.1 =>     /opt/ISV/weblib/libX.so.1
      libY.so.1 =>     /opt/ISV/weblib/libY.so.1
      libZ.so.1 =>     /opt/ISV/weblib/libZ.so.1
      libX.so.1 =>     /opt/ISV/lib/libX.so.1
      libY.so.1 =>     /opt/ISV/lib/libY.so.1

Notice, with Solaris 9 we seem to have gained two new dependencies from the directory /opt/ISV/lib.

In a previous posting I'd discussed some warnings in regard to using LD_LIBRARY_PATH, and how using a runpath was a better alternative. This customers application and dependencies are using runpaths, however the runpaths are not consistent, and are revealing the different behavior between Solaris 8 and Solaris 9.

In Solaris 8 and prior releases, dependencies were loaded by:

• First, determining whether the dependency name had already been loaded. This comparison used the NEEDED entry, in this case:

    $ elfdump -d /opt/ISV/weblib/libY.so.1
    ....
        [1]  NEEDED    0x122ef      libX.so.1
  

  This name is a simple filename, and thus gets matched against the libX.so.1 that has already been loaded as a dependency of app, as /opt/ISV/weblib/libX.so.1.

• Had this filename comparison failed, ld.so.1 would then use any search paths to try and locate the file.

It was discovered that this dependency name pattern matching was becoming a significant bottleneck, especially as the number of application dependencies continues to increase. A second drawback to this model was that requirements started to materialize for processes to be able to open different dependencies. That is, the same filename, but where the files were located in different directories.

These observations resulted in a change to the loading behavior. With Solaris 9 we no longer carry out the filename dependency pattern match against previously loaded objects. We simply search for the file using any search paths relevant to the caller (which includes the RPATH of the caller). Should this search result in a file that has already been loaded, a quick dev/inode check catches this, and prevents a duplicate loading.

The result is a much faster, and scalable search for dependencies, and the flexibility required to locate the same filename in different locations.

Hence, starting with Solaris 9 we now see:

  $ ldd -s ./app
  ....
  find object=libX.so.1; required by /opt/ISV/weblib/libY.so.1
    search path=/opt/ISV/lib:/opt/ISV/lib/../SS:......:/usr/lib/lwp \\
        (RPATH from file /opt/ISV/weblib/libY.so.1)
    trying path=/opt/ISV/lib/libX.so.1
      libX.so.1 =>     /opt/ISV/lib/libX.so.1

This search path, initiated from libY.so.1, is \*different\* from the path search that originated from ./app, and we are therefore finding two different versions of the file with the same name.

Whether two different versions of the same file are required in this users hierarchy are still unknown, perhaps they should be consolidated. But, if you want to insure the dependencies located by your components are the same, the search paths (RPATHS - set using ld -R) for all components of your system should be the same.

Published Elsewhere

[14] Relocations Don't Fit

Blog Index (rie)

[16] a source tour