Symbol Capabilities

Rod Evans — Monday August 02, 2010

Surfing with the Linker-Aliens

In a previous posting I covered the use of filters, especially defining symbol specific filtering. Filters allows the redirection of a binding at runtime to an alternative filtee. This technique has been used to provide optimized, platform specific, instances of functions. For example, has provided a number of platform specific filtees,, that provide optimized versions of the memmove() family. ldd(1) reveals these filtees.

  % ldd /bin/date =>     /lib/ =>     /lib/

In the same posting, I also touched on the use of Hardware Capabilities as a means of identifying the requirements of a filtee, and how these can be employed to provide a filtering mechanism.

Object filters do have some downsides. There's overhead involved in locating the filtees, and often a maintenance burden of providing a complex symlink hierarchy to provide the specific filtee instances.

The Solaris link-editors have now been updated to provide for multiple instances of a function to exist within the same dynamic object. Each instance of the function is associated with a group of capabilities. During process execution, the runtime linker can select from a family of symbol instances, the one whose capabilities are best represented by the present system.

This new mechanism, termed Symbol Capabilities, provides filtering within the same object, in contrast to the traditional mechanism that provides filtering by selecting from a collection of external objects.

Symbol capabilities have the advantage of providing a more light weight infrastructure than the existing filtee objects. The runtime cost of searching for, and selecting the best function, is less than searching for external objects. In addition, there's no need to maintain any symlink hierarchy to provide the specific filtee instances. Symbol capabilities can also replace the ad-hoc techniques users have employed to vector to their own family of function interfaces.

Symbol capabilities have been used to re-architect the filtering mechanism of many Solaris shared objects, including on SPARC. For example, now contains the following function instances.

  % elfdump -H /lib/

  Capabilities Section:  .SUNW_cap

   Symbol Capabilities:
       index  tag               value
         [1]  CA_SUNW_ID       sun4u
         [2]  CA_SUNW_MACH     sun4u

       index    value      size      type bind oth ver shndx   name
         [1]  0x000f0940 0x000000bc  FUNC LOCL  D    0 .text   memmove%sun4u
         [7]  0x000f1e0c 0x00001b28  FUNC LOCL  D    0 .text   memcmp%sun4u
        [11]  0x000f09fc 0x00001280  FUNC LOCL  D    0 .text   memcpy%sun4u
        [17]  0x000f1c80 0x0000018c  FUNC LOCL  D    0 .text   memset%sun4u

   Symbol Capabilities:
       index  tag               value
         [4]  CA_SUNW_ID       sun4u-opl
         [5]  CA_SUNW_PLAT     SUNW,SPARC-Enterprise

       index    value      size      type bind oth ver shndx   name
         [2]  0x000f3940 0x00000310  FUNC LOCL  D    0 .text   memmove%sun4u-opl
         [8]  0x000f458c 0x00000120  FUNC LOCL  D    0 .text   memcmp%sun4u-opl
        [12]  0x000f3c80 0x0000076c  FUNC LOCL  D    0 .text   memcpy%sun4u-opl
        [18]  0x000f4400 0x0000018c  FUNC LOCL  D    0 .text   memset%sun4u-opl

Each of these functions provides an optimized instance that is associated to a specific system. Note, that the capability identifiers have been expanded from the original hardware capabilities (CA_SUNW_HW_1) and software capabilities (CA_SUNW_SF_1) definitions, and now provide for platform identifiers (CA_SUNW_PLAT) and machine identifiers (CA_SUNW_MACH). In addition, the capabilities group can be labeled with its own identifier (CA_SUNW_ID), which in turn is used to name the function instances.

There still must exist generic interfaces for each symbol family, so contains a basic memmove(), memcpy(), etc.

      [1569]  0x000433e4 0x000001f4  FUNC GLOB  D   41 .text   memmove
      [1860]  0x000431cc 0x000001b0  FUNC GLOB  D   41 .text   memcpy

However, these generic symbols lead the family of instances, which are maintained as a chain within the capabilities data structures. During process execution, the runtime linker traverses the chain of family instances and selects the best instance for the present system.

  % elfdump -H /lib/
  Capabilities Chain Section:  .SUNW_capchain

   Capabilities family: memmove
     chainndx  symndx      name
            1  [1569]      memmove
            2  [1]         memmove%sun4u
            3  [2]         memmove%sun4u-opl
            4  [3]         memmove%sun4u-us3-hwcap1
            5  [4]         memmove%sun4u-us3-hwcap2
            6  [5]         memmove%sun4v-hwcap1
            7  [6]         memmove%sun4v-hwcap2

At runtime, you can observe which family instance is used.

  % LD_DEBUG=cap foo
  14507:   symbol=memmove[1569]:  capability family default
  14507:   symbol=memmove%sun4u[1]:  capability specific (CA_SUNW_MACH):  [ sun4u ]
  14507:   symbol=memmove%sun4u[1]:  capability candidate
  14507:   symbol=memmove%sun4u-opl[2]:  capability specific (CA_SUNW_PLAT):  [ SUNW,SPARC-Enterprise ]
  14507:   symbol=memmove%sun4u-opl[2]:  capability rejected
  14507:   symbol=memmove%sun4u-us3-hwcap1[3]:  capability specific (CA_SUNW_PLAT):  [ SUNW,Sun-Blade-1000 ]
  14507:   symbol=memmove%sun4u-us3-hwcap1[3]:  capability candidate
  14507:   symbol=memmove%sun4u-us3-hwcap1[3]:  used

In the future, it should be possible for a compiler to create various instances of a function and pass these to the link-editor. By embedding the necessary capabilities information in the object file created, the link-editor can create the necessary symbol families, and the appropriate information to provide to the runtime linker.

For now, it is also possible to create various instances using the link-editor directly. First, you need to identify a relocatable object with the capabilities it requires. This can be achieved by linking a relocatable object with a mapfile. In the following example, foo.c is built to use, and identify its requirement on, a sun4v system.

  % cc <options to trigger sun4v specific optimizations> -c -o foo.o foo.c
  % cat mapfile-cap
  $mapfile_version 2

  CAPABILITY sun4v {
          MACHINE = sun4v;

  % ld -r -o objcap.o -Mmapfile -Breduce foo.o
  % elfdump -H objcap.o

  Capabilities Section:  .SUNW_cap

   Object Capabilities:
       index  tag               value
         [0]  CA_SUNW_ID       sun4v
         [1]  CA_SUNW_MACH     sun4v

  % elfdump -s pics/objcap.o | fgrep foo
        [87]  0x00000000 0x000000bc  FUNC GLOB  D    1 .text   foo
        [93]  0x00000120 0x000000a0  FUNC GLOB  D    1 .text   bar

This capabilities object can then be translated so that each global symbol is associated with the defined capabilities.

  % ld -r -o symcap.o -z symbolcap objcap.o
  % elfdump -H symcap.o

  Capabilities Section:  .SUNW_cap

   Symbol Capabilities:
       index  tag               value
         [1]  CA_SUNW_ID       sun4v
         [2]  CA_SUNW_MACH     sun4v

       index    value      size      type bind oth ver shndx   name
        [87]  0x00000000 0x000000bc  FUNC LOCL  D    0 .text   foo%sun4v
        [93]  0x00000120 0x000000a0  FUNC LOCL  D    0 .text   bar%sun4v

Note, that this translation converts each global symbol to a local symbol, renames the symbol using the capabilities identifier, and leaves a symbol reference to the original symbol.

        [87]  0x00000000 0x000000bc  FUNC LOCL  D    0 .text   foo%sun4v
        [93]  0x00000120 0x000000a0  FUNC LOCL  D    0 .text   bar%sun4v
       [101]  0x00000000 0x00000000  FUNC GLOB  D    0 UNDEF   foo
       [102]  0x00000000 0x00000000  FUNC GLOB  D    0 UNDEF   bar

This object can now be combined into a final dynamic object together with a generic instance of the symbol family to lead the capabilities family.

Debugging Capabilities

Along with these capabilities updates, the runtime linker has also been enhanced to provide an environment for capabilities experimentation. If the previous examples were combined into a shared object,, and this object is executed on a sun4v system, then the foo%sun4v() instance is be bound to and used at runtime. However, you can establish an alternative capabilities environment, by removing or setting capabilities, along with identifying the object to which the alternative capabilities should be applied.

To exercise the generic version of foo() from, while executing on a sun4v platform, you can set the following environment variables.

  %  LD_CAP_MACH= <app>

The use of LD_CAP_FILES isolates the alternative capabilities to the object identified, rather than to every object within the process. With this mechanism, you can exercise a family of capability instances on a machine that provides all required capabilities.

Surfing with the Linker-Aliens

Published Elsewhere

Surfing with the Linker-Aliens

[29] Direct Binding...options, and probing
Blog Index (rie)
[31] elfdiff