[M3devel] strange errors...
Mika Nystrom
mika at async.caltech.edu
Wed Jul 4 14:10:39 CEST 2007
Hi again!
I was able to boot the CVS head of CM3 on my Mac just fine now,
using the cm3.cfg you sent me. I am still running the old OS
(Panther, is it?) so I tried doing it the hard way instead of using
your binaries, and I got something that works pretty well. Mentor
and Juno work, at least, and that's usually a good indicator that
one is on the right track...
But (of course there's a "but"), I am still falling down on my very
first reported bug, namely this local ktok program. I still, after
all that you've done, find that it dies on an illegal instruction
on the Macintosh (I am running 10.3 on a Powerbook).
The problem goes away if I use either @M3nogc or @M3noincremental.
The gdb backtrace is not very helpful: it says the program is
crashing somewhere in a text processing module, creating an iterator
on a sorted table.
Sorry to do this but I am feeling a bit clueless, so if you're
interested and have the time to look into it, I have set up a minimal
test here:
http://www.async.caltech.edu/~mika/ktok_example.tgz
Unfortunately, ktok itself depends on a bunch of packages.
To build this, set your CM3 environment variable to anything (1
works) and just "make" (GNU make, please). Set the DEBUGLEVEL
environment variable to "10" for a bit more output (not much). The
very last step (compiling package cit_parse) fails for me...
Specifically, the step is:
/Users/mika/t/parserlib/ktok/PPC_DARWIN/tok ../src/PRS.t -o PRSTok.i3
WELCOME!
GOT TOKPARAMS!
GOT TOKENS
GOT SUBS!
"/Users/mika/t/parserlib/parserlib/src/parser.tmpl", line 20: quake runtime error: exit 4: /Users/mika/t/parserlib/ktok/PPC_DARWIN/tok ../src/PRS.t -o PRSTok.i3
Looking a bit closer:
[lapdog:~/t/cit_parse/src] mika% gdb /Users/mika/t/parserlib/ktok/PPC_DARWIN/tok
GNU gdb 5.3-20030128 (Apple version gdb-330.1) (Fri Jul 16 21:42:28 GMT 2004)
Copyright 2003 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "powerpc-apple-darwin".
Reading symbols for shared libraries .. done
(gdb) run ../src/PRS.t -o PRSTok.i3
Starting program: /Users/mika/t/parserlib/ktok/PPC_DARWIN/tok ../src/PRS.t -o PRSTok.i3
Reading symbols for shared libraries . done
WELCOME!
GOT TOKPARAMS!
GOT TOKENS
GOT SUBS!
Program received signal EXC_BAD_INSTRUCTION, Illegal instruction/operand.
0x00b111c8 in ?? ()
(gdb) where
#0 0x00b111c8 in ?? ()
#1 0x000120e4 in TextSubs__Apply (M3_CN69dV_self=0xb26434, M3_Bd56fi_src=0xb40be0) at TextSubs.m3:64
#2 0x0005b15c in RTLinker__RunMainBody (M3_DjPxE3_m=0xad190) at RTLinker.m3:399
#3 0x00059f20 in RTLinker__AddUnitI (M3_DjPxE3_m=0xad190) at RTLinker.m3:113
#4 0x0005a01c in RTLinker__AddUnit (M3_DjPxE5_b=0x3520) at RTLinker.m3:122
#5 0x00001ecc in main (argc=4, argv=0xbffffadc, envp=0xbffffaf0) at _m3main.mc:4
(gdb)
Also:
PROCEDURE Apply(self: T; src: TEXT): TEXT =
VAR
wr := TextWr.New();
c: CHAR;
ind, pos, lastFlushed: INTEGER := 0;
textLen := Text.Length(src);
iter: SortedTextTextTbl.Iterator;
original, replacement: TEXT;
key, prefix: TEXT;
PROCEDURE Flush() =
BEGIN
Wr.PutText(wr, Text.Sub(src, lastFlushed, pos - lastFlushed));
END Flush;
BEGIN
WHILE pos < textLen DO
c := Text.GetChar(src, pos);
IF c IN self.starts THEN
Debug.Out("analysing: " & Text.Sub(src, pos), 100);
iter := self.tbl.iterateOrdered(); (* line 64 *)
ind := pos;
original := "";
REPEAT
INC(ind);
...
I am mystified as to why gdb would say that RTLinker__RunMainBody
would be calling TextSubs__Apply.
With @M3nogc or @M3noincremental, all is well...
I will see if I can figure out more, but this problem just doesn't
seem to want to go away. To the best of my knowledge, ktok doesn't
use any UNSAFE code (outside of whatever's in libm3 of course).
Mika
Tony Hosking writes:
>Sounds like we really need some work done in this area. I very
>rarely use the build scripts, since I bootstrap manually from old
>compilers to new compilers rather than use the scripts. I suggest
>the following approach, which I hope you will try, and then report
>back on.
>
>Install the bootstrap compiler as you did originally:
>
>> rm -rf /usr/local/cm3/*
>>
>> cd ~/cm3-cvs
>> mkdir boot
>> cd boot
>> tar xzvf ../cm3-min-POSIX-FreeBSD4-d5.3.1-2005-10-05.tgz
>> ./cminstall
>
>Now you will have some kind of cm3 installed, presumably in /usr/
>local/cm3/bin/cm3.
>
>Make sure you have a fresh CVS checkout in directory cm3 (let's
>assume this is in your home directory ~/cm3). Also, make sure you
>have an up-to-date version of the CM3 backend compiler cm3cg
>installed by executing the following:
>
>STEP 0:
>
>export CM3=/usr/local/cm3/bin/cm3
>cd ~/cm3/m3-sys/m3cc
>$CM3
>$CM3 -ship
>
>You can skip this last step if you know your backend compiler is up
>to date.
>
>Now, let's build the new compiler from scratch (this is the sequence
>I use regularly to test changes to the run-time system whenever I
>make them):
>
>STEP 1:
>
>cd ~/cm3/m3-libs/m3core
>$CM3
>$CM3 -ship
>cd ~/cm3/m3-libs/libm3
>$CM3
>$CM3 -ship
>cd ~/cm3/m3-sys/m3middle
>$CM3
>$CM3 -ship
>cd ~/cm3/m3-sys/m3linker
>$CM3
>$CM3 -ship
>cd ~/cm3/m3-sys/m3front
>$CM3
>$CM3 -ship
>cd ~/cm3/m3-sys/m3quake
>$CM3
>$CM3 -ship
>cd ~/cm3/m3-sys/cm3
>$CM3
>$CM3 -ship
>
>At this point you should have a bootstrapped version of cm3 installed
>in the directory /usr/local/cm3/pkg/cm3/TARGET/cm3 (where TARGET is
>the CM3 target platform you are building on -- e.g., LINUXLIBC6,
>PPC_DARWIN, ...). Note that this did not overwrite your original
>installed compiler in /usr/local/cm3/bin/cm3. We
>are now going to test the new compiler, which was built using the old
>compiler, by bootstrapping it one more time.
>
> From here on out, please replace TARGET with your target platform as
>appropriate.
>
>STEP 2:
>
>export CM3=/usr/local/cm3/pkg/cm3/TARGET/cm3
>cd ~/cm3/scripts
>./do-cm3-std.sh realclean
>REPEAT STEP 1 to rebuild the libraries and the compiler using the
>STEP 1 bootstrap compiler.
>
>Now you have a STEP 2 bootstrap compiler installed in /usr/local/cm3/
>pkg/cm3/TARGET/cm3. Let's assume the new compiler now works properly
>since it successfully bootstrapped itself, but to be sure we can now
>use it to rebuild the world:
>
>cd ~/cm3/scripts
>./do-cm3-std.sh realclean
>./do-cm3-std.sh buildship
>
>Assuming this succeeded then we can be pretty sure /usr/local/cm3/pkg/
>cm3/TARGET/cm3 is good, so we can make it our default compiler by
>replacing the original /usr/local/cm3/bin/cm3:
>
>cp $CM3 /usr/local/cm3/bin/cm3
>
>On Jun 23, 2007, at 2:38 PM, Mika Nystrom wrote:
>
>> Ok, I'm sorry if I seem a bit dimwitted in the morning like this,
>> but how exactly does one get started? I wish there were a file called
>> "GETTING_STARTED" or something like that in scripts...
>>
>> My Mac is very slow so I switched to my FreeBSD/i386 system (which has
>> PM3 happily installed on it) and tried to install CM3 from scratch.
>> Here are the exact commands I typed.
>>
>>
>> rm -rf /usr/local/cm3/*
>>
>> cd ~/cm3-cvs
>> mkdir boot
>> cd boot
>> tar xzvf ../cm3-min-POSIX-FreeBSD4-d5.3.1-2005-10-05.tgz
>> ./cminstall
>>
>> # now I seem to have some kind of cm3 installed, /usr/local/cm3/bin/
>> cm3
>>
>> cd cm3 # location of my CM3 checkout
>> cvs update -d .
>>
>> cd scripts
>> ./boot-cm3-with-m3.sh realclean
>> ./do-cm3-std.sh realclean
>>
>> ./upgrade.sh # fails here, libm3 not compiled
>> ./boot-cm3-with-m3.sh build # builds cm3, but fails on
>> cminstall, patternmatching not built
>>
>> ./do-cm3-std.sh build # OK
>> ./do-cm3-std.sh buildship # OK
>>
>> ./boot-cm3-with-m3.sh realclean # start over
>> ./boot-cm3-with-m3.sh build # OK
>> ./boot-cm3-with-m3.sh buildship # OK
>>
>> ./do-cm3-std.sh realclean # OK
>> ./do-cm3-std.sh build # dies miserably... maybe the
>> compiler can't handle the new libraries?
>>
>> ./install-cm3-compiler.sh upgrade # OK, new cm3 binary installed
>>
>> After all that, "game over." I have a cm3 that segfaults.
>>
>> Text.i3: In function 'Text_I3':
>> Text.i3:81: internal compiler error: Segmentation fault
>> Please submit a full bug report,
>> with preprocessed source if appropriate.
>> See <URL:http://gcc.gnu.org/bugs.html> for instructions.
>> compilation failed => not building library "libm3core.a"
>> Fatal Error: package build failed
>> *** execution of failed ***
>>
>> I can't seem to get either m3gdb or gdb to say anything reasonable
>> either. ktrace shows nothing out of the ordinary.
>>
>> -----
>>
>> Meanwhile, my Mac got through "do-cm3-std.sh realclean" and
>> "do-cm3-std.sh buildship" but my compiles are still dying on the
>> same assertion tickled by ktok. On that machine I have INSTALLROOT
>> set to ~/cm3, but hopefully that has nothing to do with it...
>>
>> -----
>>
>> Does do-cm3-std.sh realclean clean EVERYTHING? If so my Mac should
>> really have a fresh setup. The FreeBSD, I am not sure, maybe the
>> old binary version just doesn't work right? Of course I could try
>> bootstrapping with PM3 as well... but really, there's an "approved"
>> process to get from a blank system, no?
>>
>> Mika
>>
>>
>>
>>
>> Tony Hosking writes:
>>> That sounds like you forgot to execute "do-cm3-std.sh realclean"
>>> before initiating the build. These sorts of errors sometimes arise
>>> if there are old build directories lying around.
>>>
>>>
>>> On Jun 23, 2007, at 3:34 AM, Mika Nystrom wrote:
>>>
>>>> Hello everyone,
>>>>
>>>> I am in the process of trying to consolidate build systems for a
>>>> few software packages I am maintaining. Currently, I am using an
>>>> old PM3 on FreeBSD4, an ancient PM3 (from Klagenfurt?) for Windows
>>>> (NT386GNU), and trying to get the latest CM3 from cvs up and
>>>> compiling
>>>> things on PPC_DARWIN. Ideally, I'd like to standardize everything
>>>> on the new PM3---mainly so that I can use pickles (and Network
>>>> Objects) across all three systems. I'd also like to add Linux to
>>>> the mix.
>>>>
>>>> It's natural for me to start with CM3 on Darwin, as there's no
>>>> alternative. But I am getting some assertions failing. Everything
>>>> in the CM3 distribution compiles fine, and I believe I have compiled
>>>> the libraries a few times (that is, including with themselves), and
>>>> updated the compiler, too (using boot-cm3-with-m3.sh). I just cvs
>>>> updated tonight.
>>>>
>>>> Here's what I'm running into:
>>>>
>>>> /Users/mika/t/parserlib/ktok/PPC_DARWIN/tok ../src/CHP.t -o
>>>> CHPTok.i3
>>>>
>>>>
>>>> ***
>>>> *** runtime error:
>>>> *** <*ASSERT*> failed.
>>>> *** file "../src/runtime/common/RTCollector.m3", line 2314
>>>> ***
>>>>
>>>> Abort
>>>>
>>>> Also:
>>>>
>>>> /Users/mika/t/parserlib/ktok/PPC_DARWIN/tok ../src/PRS.t -o
>>>> PRSTok.i3
>>>> Illegal instruction
>>>>
>>>> As you can see, these things are coming from the caltech_parser. I
>>> am using
>>>> our local version, but I don't think it is very different from what
>>>> is in the
>>>> CM3 tree.
>>>>
>>>>
>>>> Examining the first error (the failed assertion) more closely, I see
>>>> the following:
>>>>
>>>> (gdb) list
>>>> 108 wr := FileWr.Open(Pathname.ReplaceExt(tp.out, "m3"));
>>>> 109 EXCEPT OSError.E =>
>>>> 110 Debug.Error("Cannot open tok implementation output
>>>> file: " &
>>>> 111 Pathname.ReplaceExt(tp.out, "m3"));
>>>> 112 END;
>>>> 113 Wr.PutText(wr, subs.apply(Bundle.Get(Form,
>>>> "tokform.m3")));
>>>> 114 Wr.Close(wr);
>>>> 115 END Main.
>>>> (gdb) where
>>>> #0 0x9004308c in kill ()
>>>> #1 0x9009fb3c in abort ()
>>>> #2 0x00096f50 in RTOS__Crash () at RTOS.m3:20
>>>> #3 0x0005bd40 in RTProcess__Crash (M3_Bd56fi_msg=0x0) at
>>>> RTProcess.m3:65
>>>> #4 0x0008e4e0 in RTError__EndError (M3_AicXUJ_crash=1 '\001') at
>>>> RTError.m3:115
>>>> #5 0x0008e08c in RTError__MsgS (M3_AJWxb1_file=0xc63d8,
>>>> M3_AcxOUs_line=2314, M3_Bd56fi_msgA=0xca3d0,
>>>> M3_Bd56fi_msgB=0xcbe90, M3_Bd56fi_msgC=0xca3d0) at RTError.m3:40
>>>> #6 0x0008eb88 in RTException__Crash (M3_Cblw37_a=0xbfffee00,
>>>> M3_AicXUJ_raises=0 '\0', M3_AJWxb1_rte=0xcb538) at RTException.m3:79
>>>> #7 0x0008e74c in RTException__DefaultBackstop
>>>> (M3_Cblw37_a=0xbfffee00, M3_AicXUJ_raises=0 '\0') at
>>>> RTException.m3:39
>>>> #8 0x0008e614 in RTException__InvokeBackstop
>>>> (M3_Cblw37_a=0xbfffee00, M3_AicXUJ_raises=0 '\0') at
>>>> RTException.m3:25
>>>> #9 0x00095d04 in RTException__Raise (M3_Cblw37_act=0xbfffee00) at
>>>> RTExFrame.m3:29
>>>> #10 0x0008e840 in RTException__DefaultBackstop
>>>> (M3_Cblw37_a=0xbfffee00, M3_AicXUJ_raises=0 '\0') at
>>>> RTException.m3:47
>>>> #11 0x0008e614 in RTException__InvokeBackstop
>>>> (M3_Cblw37_a=0xbfffee00, M3_AicXUJ_raises=0 '\0') at
>>>> RTException.m3:25
>>>> #12 0x00095d04 in RTException__Raise (M3_Cblw37_act=0xbfffee00) at
>>>> RTExFrame.m3:29
>>>> #13 0x00079740 in RTHooks__ReportFault (M3_AJWxb1_module=0xb3eb8,
>>>> M3_AcxOUs_info=74048) at RTHooks.m3:110
>>>> #14 0x0006ff4c in _m3_fault (M3_AcxOUs_arg=74048)
>>>> #15 0x0006bcf4 in RTHooks__CheckStoreTraced
>>>> (M3_Af40ku_ref=0xb2415c) at RTCollector.m3:2314
>>>> #16 0x000700e4 in ThreadPThread__InnerLockMutex
>>>> (M3_AYIbX3_m=0xb2415c, M3_BXP32l_self=0xb24014) at
>>>> ThreadPThread.m3:126
>>>> #17 0x000704d8 in ThreadPThread__LockMutex (M3_AYIbX3_m=0xb2415c)
>>>> at ThreadPThread.m3:153
>>>> #18 0x00019b24 in Wr__PutText (M3_BxxOH1_wr=0xb2415c,
>>>> M3_Bd56fi_t=0xb44d5c) at Wr.m3:93
>>>> #19 0x00003f74 in Main_M3 (M3_AcxOUs_mode=1) at Main.m3:113
>>>> #20 0x0005b1c4 in RTLinker__RunMainBody (M3_DjPxE3_m=0xad190) at
>>>> RTLinker.m3:399
>>>> #21 0x00059f88 in RTLinker__AddUnitI (M3_DjPxE3_m=0xad190) at
>>>> RTLinker.m3:113
>>>> #22 0x0005a084 in RTLinker__AddUnit (M3_DjPxE5_b=0x3600) at
>>>> RTLinker.m3:122
>>>> #23 0x00001fac in main (argc=4, argv=0xbffffb24, envp=0xbffffb38)
>>>> at _m3main.mc:4
>>>> (gdb)
>>>>
>>>> The second error:
>>>>
>>>> Program received signal EXC_BAD_INSTRUCTION, Illegal instruction/
>>>> operand.
>>>> 0x00b111ac in ?? ()
>>>> (gdb) where
>>>> #0 0x00b111ac in ?? ()
>>>> #1 0x0001214c in TextSubs__Apply (M3_CN69dV_self=0xb26450,
>>>> M3_Bd56fi_src=0xb21cec) at TextSubs.m3:63
>>>> #2 0x0005b1c4 in RTLinker__RunMainBody (M3_DjPxE3_m=0xad190) at
>>>> RTLinker.m3:399
>>>> #3 0x00059f88 in RTLinker__AddUnitI (M3_DjPxE3_m=0xad190) at
>>>> RTLinker.m3:113
>>>> #4 0x0005a084 in RTLinker__AddUnit (M3_DjPxE5_b=0x3600) at
>>>> RTLinker.m3:122
>>>> #5 0x00001fac in main (argc=4, argv=0xbffffb24, envp=0xbffffb38)
>>>> at _m3main.mc:4
>>>> (gdb) list
>>>> 58 BEGIN
>>>> 59 WHILE pos < textLen DO
>>>> 60 c := Text.GetChar(src, pos);
>>>> 61 IF c IN self.starts THEN
>>>> 62 (* S("analysing: " & Text.Sub(src, pos),
>>>> DebugLevel); *)
>>>> 63 iter := self.tbl.iterateOrdered();
>>>> 64 ind := pos;
>>>> 65 original := "";
>>>> 66 REPEAT
>>>> 67 INC(ind);
>>>> (gdb)
>>>>
>>>> Any ideas what to look at?
>>>>
>>>> I don't know if this is relevant:
>>>>
>>>> [lapdog:~/t/cit_parse/PPC_DARWIN] mika% uname -a
>>>> Darwin lapdog.local 7.9.0 Darwin Kernel Version 7.9.0: Wed Mar 30
>>>> 20:11:17 PST 2005; root:xnu/xnu-517.12.7.obj~1/RELEASE_PPC Power
>>>> Macintosh powerpc
>>>> [lapdog:~/t/cit_parse/PPC_DARWIN] mika% gcc -v
>>>> Reading specs from /usr/libexec/gcc/darwin/ppc/3.3/specs
>>>> Thread model: posix
>>>> gcc version 3.3 20030304 (Apple Computer, Inc. build 1666)
>>>>
>>>> Mika
>>>>
>>>> P.S. Am I correct in assuming that I can get CM3 to build on
>>>> Windows?
>>>> I could switch to CM3 on Unix any time, of course, but I've never
>>>> had luck with it on Windows, which is why I am using the ancient
>>>> Klagenfurt PM3, which comes on 40-odd "floppies" and a .EXE that
>>>> unpacks them into C: (and installation instructions only in German).
>>>> If CM3 doesn't work on Windows, I am essentially wasting my time,
>>>> as the current project I am working on absolutely requires that the
>>>> software run on Windows 2003 Server (yes, it sucks, but what can
>>>> you do?) The very old PM3 at least kind of hobbles along on this
>>>> platform---actually, better than that; it does Trestle natively
>>>> under Windows (no X required), at least on SOME Windows machines.
>>>>
>>>> P.P.S. Sorry for all the postscripts, but is it true that the
>>>> build system has been changed so that building with overrides
>>>> (cm3 -x)
>>>> requires having the compiler sources available? It didn't use to
>>>> be that way, but I can't get Network Objects to work with overrides
>>>> now unless I have the sources available... It's a bit of a pain
>>>> because it means that every user has to have the compiler sources,
>>>> or else ship everything, or was that not the intention?
More information about the M3devel
mailing list