#47 Library path priority plight
Opened 2 years ago by mjo. Modified 2 weeks ago

(A follow-up to what we discussed on IRC)

During linking, slibtool converts any libtool archive (*.la) arguments into -L<path> -l<library> flags. For example, when building eclib,

rlibtool --tag=CXX --mode=link ... -o solve_conic ../libsrc/libec.la

produces the linker command,

x86_64-pc-linux-gnu-g++ ... -L../libsrc/.libs -lec -o .libs/solve_conic

Compare with GNU libtool, which transforms the *.la file argument into a *.so (on Linux, anyway):

x86_64-pc-linux-gnu-g++ ... ../libsrc/.libs/libec.so -o .libs/solve_conic

The slibtool approach is more elegant, because it avoids encoding the (platform-specific) shared library extension on the command-line, but it's not equivalent in all cases to what was intended. Or more to the point: it's not equivalent to what GNU libtool does.

Specifically, -L<path> -l<library> will use the library search path to find <library>, and the priority given to <path> depends on where in the command-line -L<path> appears. If the user has other -L flags in his LDFLAGS -- or if the build system puts them there -- then it's possible that another copy of <library> will be picked up from one of those locations that has a higher precedence than <path>.

For a concrete example, the ax_boost_base.m4 macro from the autoconf-archive will add the system's library location to the command-line explicitly as e.g. -L/usr/lib64. When that happens, the transformed command gives priority to /usr/lib64/<library> over the copy in <path> that was just built. This can result in build failures if the existing, installed library has dependencies that are incompatible with the one just built. And presumably it could cause test failures in the package regardless.

Suggested fix: if possible, slibtool could rearrange the -L flags to ensure that -L<path> is given priority over any other -L flags before passing them to the linker.


Metadata Update from @midipix:
- Issue assigned to midipix

2 weeks ago

Thanks for the detailed and insightful report!

In a world not yet ideal this could (and probably should) be addressed on slibtool's side. But before we take the plunge -- ax_boost_base.m4, seriously? Hard-coding a system library path before everything else is wrong on multiple levels and should definitely be fixed upstream... Nonetheless:

  • the first possible solution is to do exactly as you have suggested, however that might get a little bit tricky due to the underlying assumption, if I understand it correctly, that the package being built will -L it's newly built libraries using a path relative to the current working directly. That is almost always true, and implementation should not be too hard given that slibtool already finalizes the argument vector as the last step before executing the link command.

  • alternatively, and more surgically, we could move "up" all -L arguments that refer to locations associated with the current build, specifically based on their .libs/ suffix.

Any thoughts?

  • the first possible solution is to do exactly as you have suggested, however that might get a little bit tricky due to the underlying assumption, if I understand it correctly, that the package being built will -L it's newly built libraries using a path relative to the current working directly. That is almost always true, and implementation should not be too hard given that slibtool already finalizes the argument vector as the last step before executing the link command.

  • alternatively, and more surgically, we could move "up" all -L arguments that refer to locations associated with the current build, specifically based on their .libs/ suffix.

The plot thickens, because ...

  • you cannot move an -L argument "up" to a point where it would precede an existing -l argument since that would change linking semantics and is thus not permitted.

  • similarly, you cannot move "up" the -l argument together with the -L argument, again since that would change linking semantics and will likely cause the build to fail (as libraries need to be specified after the object files that depend on them).

Is it possible that this is just a ax_boost_base bug after all? We haven't seen any other instance where the current logic led to undesired results, and so I wonder whether the only real solution is for ax_boost_base.m4 to be fixed.

Trying to be smart and reordering flags to be seems like it could lead to unforeseen consequences.

Take this xmlsec build issue I ran into the other day. https://github.com/lsh123/xmlsec/pull/756

As described in the PR description the program was trying to link some system libraries with -Wl,-Bstatic and others with -Wl,-Bdynamic, but the former doesn't work if only dynamic libraries are installed which went unnoticed because GNU libtool discarded the intent by being clever and ordering all of the -Wl flags at the end of linker command.

While the issue described here is different I wonder if perhaps its better if this was fixed in boost.

Login to comment on this ticket.

Metadata