Recently I've been working on the Comedi ebuild in my Gentoo overlay, wrestling with the following error:

* QA Notice: The following files contain insecure RUNPATHs
*  Please file a bug about this at http://bugs.gentoo.org/
*  with the maintaining herd of the package.
* /usr/lib:/var/tmp/portage/sci-libs/comedilib-9999/work/comedilib-9999/lib/.libs usr/lib/ruby/site_ruby/1.8/i686-linux/comedi.so

While tracking this down the source of this error, I learned a lot about dynamic linking on Linux, so here's the condensed version.

RPATH, RUNPATH, and LD_LIBRARY_PATH. The current state of affairs is well summarized on the Debian wiki, which lists the library search path:

1. the RPATH binary header (set at build-time) of the library causing the lookup (if any)
2. the RPATH binary header (set at build-time) of the executable
3. the LD_LIBRARY_PATH environment variable (set at run-time)
4. the RUNPATH binary header (set at build-time) of the executable
5. /etc/ld.so.cache
6. base library directories (/lib and /usr/lib)

There was a big dust-up between Debian and libtool back in 1999 when libtool-generated RPATHs caused problems during the libc5 to libc6 transition. The mailing list discussion makes for amusing and informative reading, if you've got a spare hour or two ;). If not, you should at least read the opening post, the description of competing contracts, and Buddha Buck's description of how linking works, although I imagine things might have changed since then. The Debian / libtool compromise (don't set RPATH by default for directories in the dynamic linker search path) was implemented in libtool 1.5.2 (released in 2004, see the Debian wiki), so this is not as big an issue as it once was.

By the way, it looks like RUNPATH was added since 1999 as a version of RPATH that did not override LD_LIBRARY_PATH, which is good, since LD_LIBARY_PATH gives you a way to link against libraries in, say, /var/tmp/portage/sci-libs/comedilib-9999/work/comedilib-9999/lib/.libs to test your executable before installation.

Anyhow, issues with rpaths persist. Since it both hard to predict all installation configurations at compile time, and tools to change rpaths later on (i.e. chrpath and patchelf) aren't able to increase the size of the rpath string on Linux (they can on Solaris, because Solaris leaves a bit of padding at the end of the dynamic string table in the compiled ELF file). This means you will have trouble moving a library out of the standard library path into some out-of-the-way location. However, in the more common case of installing a library into the standard library path, chrpath is the tool you need, and it solved my comedilib QA issue.

Along the way, I ran across two other interesting posts by Diego Pettenò about not bundling libraries.

Setting RPATH and RUNPATH

Most of the time, you'll want to avoid setting RPATH and RUNPATH and just use libraries in your usual linking path. However, sometimes they are useful. For example, SimulAVR depends on the AVR-specific libbfd, which is hopefully not in your usual path. On Gentoo, crossdev installs under /usr/lib/binutils/:

$ binutils-config -c avr
avr-git
$ grep PATH /etc/env.d/binutils/avr-git 
LIBPATH="/usr/lib/binutils/avr/git"

When you link against this library, you'll want to set RUNPATH so you don't have to remember to use LD_LIBRARY_PATH every time you run simulavr. Of course, if you switch to a different binutils version (e.g. not git), you'll need to fix the RUNPATH to point to the new target (or just rebuild simulavr against the new version).

Since you're probably not calling the linker directly when you build simulavr, you'll want to set some linker flags at configure time:

$ LDFLAGS=-Wl,-rpath=/usr/lib/binutils/avr/git,--enable-new-dtags ./configure …

The relevant linker flags are -rpath and --enable-new-dtags. Without --enable-new-dtags, you'll just set the RPATH flag, which is probably not what you want. With --enable-new-dtags, you'll set both RPAH and RUNPATH to the same value. From ld(1):

The DT_RPATH entries are ignored if DT_RUNPATH entries exist.

so setting both is the same as just setting RUNPATH (except for tools like chrpath which are only designed to handle a single tag). You can use readelf to see if the tags were set:

$ readelf --dynamic /usr/bin/simulavr | grep PATH
 0x000000000000000f (RPATH)    Library rpath: [/usr/lib/binutils/avr/git]
 0x000000000000001d (RUNPATH)  Library runpath: [/usr/lib/binutils/avr/git]

On Gentoo, --enable-new-dtags has been the default since 2005, but explicitly including the flag can't hurt ;).