[M3devel] map/def files to limit exports to only functions in interfaces

[Jay]

On Windows, a "def" file lists extra linker parameters, mainly a list of functions to export.
On most Posix systems, a "map" serves the same purpose.
On Windows, a "map" file is actually a linker output, that is roughly, a text file with some symbol information.
 
 
On Windows, the default set of functions to export is empty.
You can either decorate your source code with __declspec(dllexport), or write a .def file.
Some build systems -- including cm3 -- grovel .obj files and put every symbol in the .def file.
 
 
Typical Unix behavior is to export all symbols.
 
 
More so, typical Unix behavior is to try to make "shared objects" very much like "objects".
In particular, to treat all the shared objects within a process as containing one global/pooled namespace.
 
 
Therefore, the first shared object loaded that exports a function named "open", is the target of any unresolved links to the symbol "open".
 
 
This is a feature. It lets folks "interpose" and/or "preload" their own special shared object that implements whatever function in its own special way. Such dynamic resolution occurs by default for all non-static symbols. I'm not sure how it works out, but two shared objects might contain a function foo, that they each call, expecting to their own, but as I understand, they might both call the first loaded one. You can control linking presumably by declaring functions static, but that doesn't work if you have multiple source files in your shared object and want to call between them.
 
 
There are some details I am uncertain of, to be sure.
 
 
> This is a feature..
 
 
But, I believe it is more broadly understood to be a mistake.
 
 
At some early point, like around 10.2, Apple changed from a "flat namespace" to "two level namespace".
That is, when I link, on the development machine, "open" was found in e.g. "libc.so",
so in my foo.exe, not just the symbol "open" is recorded, but "libc" is associated with it.
So at load time, "open" is not looked at in all shared objects, but only in "libc" (could there be more than one, in different directories, I don't know).
Is this just a "hint" or it "must be so"? I'm not sure.
 
 
Solaris linker man pages give the "export everything" as the historical default, but not the recommended practise.
 
 
There is a paper by the Linux ld.so maintainer -- Ulrich Drepper -- with advise I believe compatible with my thinking. Limit exports to only what you intend. Bind symbols locally that you implement. Don't leave all symbols "interposable", only ones you intend to import from somewhere else.
 
 
When I was debugging AMD64_LINUX months ago, I found that even nested functions were exported and interposable, and the delayed resolution of them trashed the static link.
So at the time I changed just their behavior.
 
 
There is new syntax and command line flags to gcc to control what they call "visibility".
Historical default, everything is visible/exported.
Current guidance is, switch the default to not exported, and "decorate" source code with "__attribute__" or a wrapper macro thereof.
 
 
SO......
 
 
Today we export "every" function -- including functions not defined in an .i3 file.
 
 
This is inefficient.
 
 
I strongly propose that we generate "def" files for Win32 and "map" files for "everyone else" (being sure to read up on the Solaris linker, GNU linker, BSD linkers, and later Irix, AIX, HP-UX, OSF, etc.) that list only functions in .i3 files, and the module_m3 and/or _i3 symbols, whatever is needed.
 
 
I've already done some initial investigation here.
It is doable in the front end but I believe m3linker is the right place.
 The front end visits interface/module at a time, whereas this work needs to be done once per "package" or "library", and cannot really be done incrementally, and it is small anyway.
 
We don't need signatures, just lists of functions (and, on some systems..variables) in interfaces.
Actually, I think this work could lead to "fixing data imports on Win32", but that's another subject and it is likely avoidable (you would change all cross-interface variable references to go through pointers, in case they are imported, then linker can synthesize the pointers if ends up not imported, kind of backwards from the way importing functions where, where the default is a direct call and linker can synthesize thunks for imports).
 
 
Reasonable?
 
 
 - Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20090112/c929d21f/attachment-0001.html>