[M3devel] New setjmp/longjmp problem.

[Rodney M. Bates]

On Mandriva 2007.0, LINUXLIBC6, all dynamically linked executables produced by
either the CM3 or PM3 compilers segfault during initialization of the runtime.
The problem  occurs in longjmp.  There is a replacement for setjmp in the cm3
distribution with this comment:

#
# Setjmp, _setjmp, and __setjmp are broken in glibc 2.0.7.
# They share code with sigsetjmp which needs 2 arguments, and
# this does not work well on the i386 where the caller must
# pop the stack for the arguments when the call returns.
# Indeed, their setjmp adds the missing argument and jumps to
# sigsetjmp. Upon return, however, the caller pops one argument
# and the second argument remains on the stack, making the stack
# pointer 4 bytes away from its correct position.
#
# Below is a very simple reimplementation of _setjmp, the only function
# used within the Modula-3 runtime.
#

The setjmp in glibc contains the line:

	xor    %gs:0x18,%ecx

at two places where the stack pointer and the return address are in %ecx.  The
replacement just stores these registers in the jmpbuf without molesting them.
Up at least through libc-2.3.4 (where this problem does not occur), longjmp
just reloads the stack and instruction pointers from the jmpbuf.  This does not
sound like the problem the comment identifies, but it clearly is inconsistent
and would cause the setjmp/longjmp pair to fail.

As of libc-2.4, the original brokenness of setjmp seems to have been fixed by
doing the very same xor operations to these values when reloading them from
the jmpbuf, in longjmp.  The fault occurs inside RTThreadC.c, function Transfer,
which calls setjmp and immediately passes the result to longjmp.  Removing the
replacement for setjmp and recompiling so that both setjmp and longjmp come from
libc makes this segfault go away.

Unfortunately, this just causes another segfault almost immediately after,
when M3 runtime code attempts to use the contents of these registers from
a jmpbuf, supplied by a different call on setjmp, where the values are now
garbled by the original xor problem with setjmp.

I am looking for suggestions on what to do.  Possibilities that have occurred
to me:

1.  Also add an assembly code replacement for longjmp too, taken from an earlier
     libc.  longjmp is quite a bit longer, and does some tricky stuff I have not
     dug through yet.

2.  Use the libc setjmp for values that are to be passed to longjmp and use the
     replacement setjmp for values that are to be used by the M3 runtime.  This
     seems cleaner in the long run, but a preliminary grep reveals massive numbers
     of hits on "setjmp".  I had been hoping there would only be a few.  The two
     uses of jmpbuf values may not even be disjoint.

This will probably affect other distributions as they go to later libc versions.
Any suggestions would be appreciated.

P.S:  I had some cases where changing the M3 compilation to link statically made
the problem go away, but now I can't reproduce them.  Instead, bash and ldd claim
"executable: No such file or directory", while "ls -l executable" and many other
commands find "executable" just fine.


-- 
-------------------------------------------------------------
Rodney M. Bates, retired assistant professor
Dept. of Computer Science, Wichita State University
Wichita, KS 67260-0083
316-978-3922
rodney.bates at wichita.edu