Shared Object Filters

Rod Evans — Friday October 22, 2004

Filters are a class of shared objects that are available with Solaris. These objects allow for the redirection of symbol bindings at runtime. They have been employed to provide standards interfaces and to allow the selection of optimized function implementations at runtime. For those of you unfamiliar with filters, here's an introduction, plus a glimpse of some new techniques available with Solaris 10.

Filters can exist in two forms, standard filters and auxiliary filters. At runtime, these objects redirect bindings from themselves to alternative shared objects, that are known as filtees. Shared objects are identified as filters by using the link-editors -F and -f options.

Standard filters provide no implementation for the interfaces they define. Essentially they provide a symbol table. When used to build a dynamic object they satisfy the symbol resolution requirements of the link-editing process. However at runtime, standard filters redirect a symbol binding from themselves to a filtee.

Standard filters have been used to implement libdl.so.1, the library that offers the dynamic linking interfaces. This library has no implementation details, it simply points to the real implementation within the runtime linker:

    % elfdump -d /usr/lib/libdl.so.1

    Dynamic Section:  .dynamic
      index  tag       value
      [0]  SONAME      0x138  libdl.so.1
      [1]  FILTER      0x143  /usr/lib/ld.so.1
      ...

Auxiliary filters work in much the same way, however if a filtee implementation can't be found, the symbol binding is satisfied by the filter itself. Basically, auxiliary filters allow an interface to find a better alternative. If an alternative is found it will be used, and if not, the generic implementation provided by the filter provides a fallback.

Auxiliary filters have been used to provide platform specific implementations. Typically, these are implementations that provide a performance improvement on various platforms. libc.so.1 has used this technique to provide optimized versions of the memcpy() family of routines:

    % elfdump -d /usr/lib/libc.so.1

    Dynamic Section:  .dynamic
      index  tag        value
      ...
      [3]  SONAME      0x6280  libc.so.1
      [4]  AUXILIARY   0x628a  /usr/platform/$PLATFORM/lib/libc_ psr.so.1
      ...

You can observe that a symbol binding has been satisfied by a filtee by using the runtime linkers tracing. The following output shows the symbol memset being searched for in the application date, the dependency libc.so.1, and then in libc's filtee, libc_psr.so.1.

    % LD_DEBUG=symbols,bindings  date
    .....
    11055: symbol=memset;  lookup in file=/usr/bin/date
    11055: symbol=memset;  lookup in file=/usr/lib/libc.so.1
    11055: symbol=memset;  lookup in file=/usr/platform/SUNW,Sun-Fire/lib/libc_psr.so.1
    11055: binding file=/usr/lib/libc.so.1 to \\
        file=/usr/platform/SUNW,Sun-Fire/lib/libc_psr.so.1: symbol `memset'
    .....

Until now, a filter has been an identification applied to a whole shared object. With Solaris 10, per-symbol filtering has been introduced. This allows individual symbol table entries to identify themselves as standard, or auxiliary filters. These filters provide greater flexibility, together with less runtime overhead than whole object filters. Individual symbols are identified as filters using mapfile entries at the time an object is built.

For example, libc.so.1 now provides a number of per-symbol filters. Each filter is defined using a mapfile entry:

   % cat mapfile
   SUNW_1.22 {
       global:
           ....
           dlopen = FUNCTION FILTER /usr/lib/ld.so.1;
           dlsym = FUNCTION FILTER /usr/lib/ld.so.1;
           ....
   };
   ....
   SUNW_0.7 {
      global:
          ....
          memcmp = AUXILIARY /platform/$PLATFORM/lib/libc_psr.so.1;
          memcpy = AUXILIARY /platform/$PLATFORM/lib/libc_psr.so.1;
          ....
   };

The above definitions provide a couple of advantages. First, you no longer need to link against libdl to obtain the dynamic linking family of routines. Second, the overhead of searching for a filtee will only occur if a search for the associated symbol is requested. With whole object filters, any reference to a symbol within the filter would trigger a filtee lookup, plus every interface offered by the filter would be searched for within the filtee.

Per-symbol filters have also proved useful in consolidating existing interfaces. For example, for historic standards compliance, libc.so.1 has offered a small number of math routines. However the full family of math routines are provided in libm.so.2, the library that most math users link with. Providing the small family of duplicate math routines in libc was a maintenance burden, plus there was always the chance of them getting out of sync. With per-symbol filtering, the libc interfaces are maintained, while pointing at the one true implementation. elfdump(1) can be used to reveal the filter symbols (F) offered by a shared object:

   % elfdump -y /lib/libc.so.1

   Syminfo Section:  .SUNW_syminfo
     index  flgs         bound to           symbol
      ....
      [93]  F        [1] libm.so.2          isnand
      [95]  F        [1] libm.so.2          isnanf
      ....

The definition of a filtee has often employed runtime tokens such as $PLATFORM. These tokens are expanded to provide pathnames specific to the environment in which the filter is found. A new capability provided with Solaris 10 is the $HWCAP token. This token is used to identify a directory in which one or more hardware capability libraries can be found.

Shared objects can be built to record their hardware capabilities requirements. Filtees can be constructed that use various hardware capabilities as a means of optimizing their performance. These filtees can be collected in a single directory. The $HWCAP token can then be employed by the filter to provide the selection of the optimal filtee at runtime:

   % elfdump -H /opt/ISV/lib/hwcap/\*

   /opt/ISV/lib/hwcap/libfoo_hwcap1.so.1:

   Hardware/Software Capabilities Section:  .SUNW_cap
     index  tag               value
       [0]  CA_SUNW_HW_1     0x869  [ SSE  MMX  CMOV  SEP  FPU ]

   /opt/ISV/lib/hwcap/libfoo_hwcap2.so.1:

   Hardware/Software Capabilities Section:  .SUNW_cap
     index  tag               value
      [0]  CA_SUNW_HW_1     0x1871  [ SSE2  SSE  MMX  CMOV  AMD_SYSC  FPU ]

   % elfdump -d /opt/ISV/lib/libfoo.so.1

    Dynamic Section:  .dynamic
      index  tag       value
      ...
      [3]  SONAME      0x138  libfoo.so.1
      [4]  AUXILIARY   0x4124 /opt/ISV/lib/hwcap/$HWCAP
      ...

The hardware capabilities of a platform are conveyed to the runtime linker from the kernel. The runtime linker then matches these capabilities against the requirements of each filtee. The filtees are then sorted in descending order of their hardware capability values. These sorted filtees are used to resolve symbols that are defined within the filter.

This model of file identification provides greater flexibility than the existing use of $PLATFORM, and is well suited to filtering use.

As usual, examples and all the gory details of filters and the new techniques outlined above can be found in the Solaris Linker and Libraries guide.

Discussions on how to alter any hardware capabilities established by the compilers can be found here, and here.

Comments

PatrickG — Saturday October 23, 2004

Rod Evans — Sunday October 24, 2004

Yes, I guess you could use a filter to implement your scenario, although I doubt we would.

For a quick "strcat" fix you could put the fixed function in a library of its own, and then make that library available to a process with <tt>LD_PRELOAD</tt>.

Filters have typically been employed to provide alternative implementations, where each implementation can take advantage of instructions specific to a particular platform. Security holes or bug fixes are usually handled by providing an update to the original library.

Michael van der Westhuizen — Sunday October 24, 2004

Published Elsewhere

[9] Tracing a link-edit

Blog Index (rie)

[11] Static Linking EOL