[M3devel] pkg roots/origin/ld_library_path

Fri Jul 10 03:36:21 CEST 2009

I think we very much agree.

Which comes first, LD_LIBRARY_PATH or runpath?

The default build uses both $ORIGIN/../lib and fullpath.
  Like you suggest.
I forget the order. $ORIGIN first would be good I think. I'll check and make it so.

There is an environment variable to suppress the fullpath.
The existing "scripts" set it.
 You can find its name in the scripts and the config files.
 Something like CM3_PORTABLE_RUNPATH.
I know environment variables are lame.

The structure has already been flattened.
It was always "available" flattened, but symlinked into the deep structure, which I believe largely defeated our current goals.
Now there is just the flat structure and the deep structure is "gone" (well, two files removed out of each directory).

I also put $ORIGIN alone in the runpath, without the ../lib, so you can further flatten lib into bin. I'm fairly sympathetic to that but I suspect not many other people are. ?

I always reserve the maximum space on MacOSX (in the config files), having seen that stuff in the manpages and figuring "reserving the maximum" probably isn't too wasteful and is a nice item on the "flexibility checklist" (that is, something I'll probably never use, but if anyone wants it, sure, it's there). That way, "link upon install" can be "run that tool upon install".

I had seen similar for Solaris but I forget the details, which you filled in.

 - Jay

----------------------------------------
> Date: Thu, 9 Jul 2009 16:15:51 -0700
> From: carson at taltos.org
> To: m3devel at elegosoft.com
> Subject: Re: [M3devel] pkg roots/origin/ld_library_path
>
> LD_LIBRARY_PATH (or DYLD_LIBRARY_PATH on MacOS X) _usually_ overrides a
> compiled-in search path. I know it does on Linux, MacOS X, and Solaris.
>
> And yes, you can "omit the directory" on most flavours of UNIX (although not on
> MacOS X, which gets complicated). If you leave out rpath, only the system
> default locations (and LD_LIBRARY_PATH, if set) will be searched.
>
> My opinion:
>
> - The primary installation should be built using a relative RPATH only
> ($ORIGIN/../lib or whatever). That does imply flatening the installed namespace
> to something more traditional, or including symlinks (if you have
> /usr/local/m3/lib/libfoo.so and /usr/local/m3/my/sub/module/mybinary you'd need
> to symlink /usr/local/m3/my/sub/module/lib to ../../../lib, or symlink each .so.*).
>
> - If a user builds with an alternate root, it should include an absolute RPATH
> for the standard installed location. It may _also_ include a relative RPATH, if
> needed (module builds its own .so libs), which should come first.
>
> This allows:
>
> - The binary install to be moved between systems easily
> - Binary non-standard location modules to be moved to any location on systems
> that share the same standard install location.
>
> This is about as flexible as you can get on most platforms.
>
> Note that on MacOS X you can use install_name_tool to re-write library
> locations, as long as the original was built with enough padding (linked with
> -headerpad_max_install_names, or the new name is shorter). This will also be
> possible on newer versions of Solaris, but is not yet in Sol 10 (only in
> OpenSolaris).
>
>
> jay.krell at cornell.edu wrote:
>> I see the base contradiction is between custom install location, non
>> root install, and Unix binary installs. Whenever I install binaries in
>> Unix I have to be root and I get no choice of install location. Only
>> when I build from source are both of these relaxed.
>>
>> Windows varies. Often I can chose install location and sometimes I don't
>> have to be admin (but I always am).
>>
>> $origin is a way to address this, but relative locations remain fixed.
>>
>> - Jay (phone)
>>
>> On Jul 7, 2009, at 6:05 AM, Olaf Wagner wrote:
>>
>>> I understand that the $ORIGIN use in library paths does not work
>>> if we want to have hierarchical overrides for an arbitrary set of
>>> package pools. We'll either have to use absolute paths or rely
>>> on a correct (automatically computed?) LD_LIBRARY_PATH setting.
>>>
>>> How do $ORIGIN and LD_LIBRARY_PATH work together? Are there
>>> consistently defined semantics on different systems?
>>>
>>> Could we use $ORIGIN just for the one global system installation,
>>> and rely on absolute paths for all other pools? Of course we'd
>>> loose the ability to just copy from one pool to another this way,
>>> but that may not be that important.
>>>
>>> Anyway, we (at least I) won't work on it right now...
>>>
>>> Olaf
>>>
>>> Quoting Jay K :
>>>
>>>>
>>>> Really, I understand, but there is a problem nobody seems to see.
>>>>
>>>>
>>>> Let's say there are multiple pkg roots.
>>>>
>>>>
>>>> Remember my question? It is very important.
>>>> "How are the dependencies of an .so or executable found?"
>>>>
>>>>
>>>> What if I install to /opt/cm3.
>>>> You install to $HOME/cm3 -- no need for root!
>>>> We have some root/bin/executable.
>>>> It uses of course libm3core.so.
>>>>
>>>>
>>>> I want to be able to copy it from machine to machine, probably
>>>> without relinking it or running some tool over it.
>>>>
>>>>
>>>> People want to install libm3core.so to different places on different
>>>> machines.
>>>>
>>>>
>>>> Does root/bin/executable refer
>>>> to /opt/cm3/lib?
>>>> or $HOME/cm3/lib?
>>>> or $ORIGIN/../lib?
>>>> or no directory and user sets LD_LIBRARY_PATH?
>>>> I'm not even sure you can omit the directory.
>>>>
>>>>
>>>> There is a basic contradiction here.
>>>> $ORIGIN is good. $ORIGIN is bad.
>>>> LD_LIBRARY_PATH is good. LD_LIBRARY_PATH is bad.
>>>> Full paths are good. Full paths are bad.
>>>> The ability to install to different places is good. The ability to
>>>> install to different places is bad.
>>>>
>>>>
>>>> You cannot have all of:
>>>> ability to specify install location
>>>> AND multiple package roots, where a package root is sparse
>>>> AND if I install to a custom location, I don't have to set
>>>> LD_LIBRARY_PATH
>>>>
>>>>
>>>> Now, while I advocate(d) $ORIGIN and custom install locations and no
>>>> need for LD_LIBRARY_PATH, that approach does devolve to everything
>>>> being in the same directory, which also seems not great.
>>>>
>>>>
>>>> The difference is often that you don't use $ORIGIN for "well known
>>>> important stuff".
>>>> You don't get "libc.so" from $ORIGIN/../lib. You get it from where
>>>> it always is, like /usr/lib or somesuch. You don't actually ever use
>>>> a custom install for it. If you want to build your own, you are
>>>> probably building your own entire system, and might use chroot as
>>>> part of that.
>>>>
>>>>
>>>> But where is the line between "my stuff" and "well known important
>>>> stuff"? Where is the line between stuff that can be installed
>>>> anywhere vs. stuff that must be installed to its "standard" location?
>>>>
>>>>
>>>> - Jay
>>>
>>>
>>>
>>> --
>>> Olaf Wagner -- elego Software Solutions GmbH
>>> Gustav-Meyer-Allee 25 / Gebäude 12, 13355 Berlin, Germany
>>> phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45
>>> 86 95
>>> http://www.elegosoft.com | Geschäftsführer: Olaf Wagner | Sitz: Berlin
>>> Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr:
>>> DE163214194
>>>
>>>
>
>