From mika at async.caltech.edu Wed Jan 1 11:48:36 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Wed, 01 Jan 2014 02:48:36 -0800 Subject: [M3devel] mklib not compiling Message-ID: <20140101104836.E13C51A208E@async.async.caltech.edu> Hi m3devel, Trying to upgrade my compiler on AMD64_LINUX... === package /home/mika/cm3-cvs-anon/cm3/m3-sys/mklib === +++ cm3 -build -DROOT='/home/mika/cm3-cvs-anon/cm3' $RARGS && cm3 -ship $RARGS -DROOT='/home/mika/cm3-cvs-anon/cm3' +++ --- building in AMD64_LINUX --- ignoring ../src/m3overrides new source -> compiling Main.m3 "../src/Main.m3", line 659: incompatible types (b) "../src/Main.m3", line 660: incompatible types (b) "../src/Main.m3", line 661: incompatible types (b) "../src/Main.m3", line 662: incompatible types (b) "../src/Main.m3", line 663: incompatible types (b) "../src/Main.m3", line 664: incompatible types (b) "../src/Main.m3", line 665: incompatible types (b) 7 errors encountered compilation failed => not building program "mklib" Fatal Error: package build failed I'm at the head... Mika From mika at async.caltech.edu Thu Jan 2 17:34:10 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Thu, 02 Jan 2014 08:34:10 -0800 Subject: [M3devel] building the CVS head---how??? Message-ID: <20140102163410.757DB1A2080@async.async.caltech.edu> So here is the situation. New machine, new problems. AMD64_LINUX. I want to try the latest compiler, because there are issues with what I have. What I have is the following: Critical Mass Modula-3 version 5.8.6 last updated: 2010-04-11 compiled: 2010-07-12 20:10:34 configuration: /home/mika/cm3.amd64-linux/bin/cm3.cfg host: AMD64_LINUX target: AMD64_LINUX I think the boot archives on the CM3 web site are about the same age. I try ./boot1.py AMD64_LINUX and I get "../src/win32/WinSock.i3", line 662: unsupported language or calling convention (PASCAL) "../src/win32/WinSock.i3", line 665: unsupported language or calling convention (PASCAL) "../src/win32/WinSock.i3", line 688: unsupported language or calling convention (PASCAL) "../src/win32/WinSock.i3", line 691: unsupported language or calling convention (PASCAL) "../src/win32/WinSock.i3", line 694: unsupported language or calling convention (PASCAL) "../src/win32/WinSock.i3", line 697: unsupported language or calling convention (PASCAL) 34 errors encountered new exporters -> recompiling WinVer.i3 "../src/win32/WinVer.i3", line 158: unsupported language or calling convention (APIENTRY) "../src/win32/WinVer.i3", line 168: unsupported language or calling convention (APIENTRY) "../src/win32/WinVer.i3", line 180: unsupported language or calling convention (APIENTRY) "../src/win32/WinVer.i3", line 190: unsupported language or calling convention (APIENTRY) "../src/win32/WinVer.i3", line 203: unsupported language or calling convention (APIENTRY) (etc) all right, it's broken, something needs to be fixed. WHAT IS NOT ALL RIGHT: At this point my regular cm3.cfg is messed up. If I had been silly enough to try to bootstrap a new compiler (for a different machine) without first saving my old CM3 installation, I'd be screwed. But from reading the mailing list logs, I am not sure if it is even possible to bootstrap the head from the older installation. How am I supposed to get a new compiler working? Build it on my Raspberry Pi (which has a working, recent CM3 on it)? (After backing up my old installation, of course.) And what's this? Bad python? (119)truffles:~/cm3-cvs-anon/cm3/scripts/python>./boot1.py AMD64_LINUX Traceback (most recent call last): File "./boot1.py", line 5, in import pylib File "/big/home2/mika/2/cm3-cvs/cm3/scripts/python/pylib.py", line 587, in for a in os.popen(CM3 + " -version 2>" + DevNull): TypeError: unsupported operand type(s) for +: 'NoneType' and 'str' (120)truffles:~/cm3-cvs-anon/cm3/scripts/python>python --version Python 2.7.3 Mika Mika From mika at async.caltech.edu Fri Jan 3 00:15:37 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Thu, 02 Jan 2014 15:15:37 -0800 Subject: [M3devel] booting for AMD64_LINUX, continued... Message-ID: <20140102231537.B953A1A2080@async.async.caltech.edu> So I'm trying to get this working by cross-compiling from the ARM_LINUX to AMD64_LINUX by doing ./boot1.py c AMD64_LINUX I get: new source -> compiling PklActionSeq.i3 new source -> compiling ConvertPacking.m3 "../src/pickle/ver2/ConvertPacking.m3", line 327: warning: not used (ReadWC21) "../src/pickle/ver2/ConvertPacking.m3", line 851: warning: not used (WriteWC21) *** *** runtime error: *** <*ASSERT*> failed. *** file "../src/M3C.m3", line 5890 *** Aborted *** execution of [] failed *** 5887 PROCEDURE shift_right(self: T; type: IType) = 5888 (* s1.type := Word.Shift (s1.type, -s0.type); pop *) 5889 BEGIN 5890 <* ASSERT cgtypeIsUnsignedInt[type] *> 5891 shift_left_or_right(self, type, "shift_right", ">>"); 5892 END shift_right; ... Does anyone have a relatively recent working AMD64_LINUX installation? I'd love a copy to try to boot from... Mika From mika at async.caltech.edu Fri Jan 3 06:12:52 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Thu, 02 Jan 2014 21:12:52 -0800 Subject: [M3devel] booting for AMD64_LINUX, continued... In-Reply-To: <20140102174632.3169C41A@resin13.mta.everyone.net> References: <20140102174632.3169C41A@resin13.mta.everyone.net> Message-ID: <20140103051252.88FC61A2080@async.async.caltech.edu> Well I managed to get it to work using the following roundabout procedure: Since the only archives I could find on elegosoft for AMD64_LINUX were ancient, I installed VirtualBox and FreeBSD/amd64 inside that. FreeBSD/amd64 for some reason has very recent archives (of the old "cminstall" kind). Downloaded the FreeBSD archive, cminstalled it, and ran upgrade.py on the latest sources. Had to remove some code from ConvertPacking.m3 because Jay's C-backend can't compile that for some reason. ran boot1.py, which gave me a boot archive. Note---if you've done this with boot1.py "c" before, you have to delete all the derived dirs and start over, or you get a mess. Moved the boot archive to the Linux machine. Resurrected ConvertPacking.m3 Ran boot2.sh (or equiv) until I got everything built. That worked. So... some observations: .. I have a very recent build on ARM_LINUX, but wasn't able to cross-compile. Word size problem? .. I now have three very recent builds: ARM_LINUX, AMD64_FREEBSD, and AMD64_LINUX. What precisely do I have to do to make boot archives out of these so other people might be spared this painful process? Very few archives on the elego site are up to date. Some NT stuff and some FreeBSD stuff. Nothing else that I can see. Mika "Rodney Bates" writes: > > >-Rodney Bates > >--- mika at async.caltech.edu wrote: > >From: mika at async.caltech.edu >To: m3devel at elegosoft.com >Subject: [M3devel] booting for AMD64_LINUX, continued... >Date: Thu, 02 Jan 2014 15:15:37 -0800 > > >So I'm trying to get this working by cross-compiling from the ARM_LINUX to AMD64_LINUX by doing > >./boot1.py c AMD64_LINUX > >I get: > >new source -> compiling PklActionSeq.i3 >new source -> compiling ConvertPacking.m3 >"../src/pickle/ver2/ConvertPacking.m3", line 327: warning: not used (ReadWC21) >"../src/pickle/ver2/ConvertPacking.m3", line 851: warning: not used (WriteWC21) > > >*** >*** runtime error: >*** <*ASSERT*> failed. >*** file "../src/M3C.m3", line 5890 >*** > >Aborted > *** execution of [] failed *** > > > 5887 PROCEDURE shift_right(self: T; type: IType) = > 5888 (* s1.type := Word.Shift (s1.type, -s0.type); pop *) > 5889 BEGIN > 5890 <* ASSERT cgtypeIsUnsignedInt[type] *> > 5891 shift_left_or_right(self, type, "shift_right", ">>"); > 5892 END shift_right; > >... > >Does anyone have a relatively recent working AMD64_LINUX installation? I'd love a copy to try to boot from... > >I have one, but am out of the country right now and have no access to it. >I am waiting as I write to board the first airplane home, so will try to >tar one up for you when I get there. > > > Mika > From mika at async.caltech.edu Fri Jan 3 06:57:18 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Thu, 02 Jan 2014 21:57:18 -0800 Subject: [M3devel] pthreads still broken Message-ID: <20140103055718.B12821A2080@async.async.caltech.edu> On my new AMD64_LINUX build... The thread tester still crashes. First, it prints a couple of extraneous diagnostics. These look like they come from the threading code? Then it segfaults. (141)truffles:~/cm3-cvs-anon/cm3/m3-libs/m3core/tests/thread/AMD64_LINUX>gdb threadtest GNU gdb (GDB) 7.4.1-debian Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/thread/AMD64_LINUX/threadtest...done. (gdb) run Starting program: /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/thread/AMD64_LINUX/threadtest [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Writing file...done Creating read threads...[New Thread 0x2aaaab6f8700 (LWP 23996)] [New Thread 0x2aaaab8f9700 (LWP 23997)] [New Thread 0x2aaaabafa700 (LWP 23998)] done Creating fork threads...[New Thread 0x2aaaabcfb700 (LWP 23999)] [New Thread 0x2aaaabefc700 (LWP 24000)] [New Thread 0x2aaaac0fd700 (LWP 24001)] done Creating alloc threads...[New Thread 0x2aaaac2fe700 (LWP 24002)] [New Thread 0x2aaaac4ff700 (LWP 24003)] [New Thread 0x2aaaac700700 (LWP 24004)] done Creating lock threads...[New Thread 0x2aaaac901700 (LWP 24005)] [New Thread 0x2aaaacb02700 (LWP 24006)] [New Thread 0x2aaaacd03700 (LWP 24007)] done running...printing oldest/median age/newest Program received signal SIG64, Real-time event 64. [Switching to Thread 0x2aaaac901700 (LWP 24005)] 0x00002aaaaaf5b64b in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0 (gdb) handle SIG64 nostop noprint Signal Stop Print Pass to program Description SIG64 No No Yes Real-time event 64 (gdb) cont Continuing. ....pthread_mutex_destroy:EBUSY ...pthread_mutex_destroy:EBUSY Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x2aaaab6f8700 (LWP 23996)] 0x0000000000416de6 in FileRd__Init (M3_DaDE31_rd=, M3_EQj1Bs_h=) at ../src/rw/FileRd.m3:44 44 IF (rd.buff = NIL) THEN (gdb) where #0 0x0000000000416de6 in FileRd__Init (M3_DaDE31_rd=, M3_EQj1Bs_h=) at ../src/rw/FileRd.m3:44 #1 0x0000000000416d3a in FileRd__Open (M3_Bd56fi_p=) at ../src/rw/FileRd.m3:16 #2 0x0000000000405424 in Main__RApply (M3_AP7a1g_cl=) at ../src/Main.m3:182 #3 0x000000000044c23b in ThreadPThread__RunThread (M3_DMxDjQ_me=) at ../src/thread/PTHREAD/ThreadPThread.m3:449 #4 0x000000000044bef8 in ThreadPThread__ThreadBase (M3_AJWxb1_param=) at ../src/thread/PTHREAD/ThreadPThread.m3:422 #5 0x00002aaaaaf56b50 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #6 0x00002aaaab246a7d in clone () from /lib/x86_64-linux-gnu/libc.so.6 #7 0x0000000000000000 in ?? () (gdb) The reason I even ran it was because I found that running the parallel compiler backend was unstable. Do user threads work on AMD64_LINUX? Mika From mika at async.caltech.edu Fri Jan 3 17:13:43 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Fri, 03 Jan 2014 08:13:43 -0800 Subject: [M3devel] thread tester results on ARM_LINUX Message-ID: <20140103161343.F25291A2080@async.async.caltech.edu> (78)raspberrypi:~/cm3-cvs-anon/cm3/m3-libs/m3core/tests/thread>gdb ARM_LINUX/threadtest GNU gdb (GDB) 7.4.1-debian Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "arm-linux-gnueabihf". For bug reporting instructions, please see: ... Reading symbols from /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/thread/ARM_LINUX/threadtest...done. (gdb) handle SIGILL Signal Stop Print Pass to program Description SIGILL Yes Yes Yes Illegal instruction (gdb) handle SIGILL nostop noprint Signal Stop Print Pass to program Description SIGILL No No Yes Illegal instruction (gdb) run Starting program: /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/thread/ARM_LINUX/threadtest [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1". Writing file...done Creating read threads...[New Thread 0x409f4450 (LWP 2941)] [New Thread 0x40bf4450 (LWP 2942)] [New Thread 0x40df4450 (LWP 2943)] done Creating fork threads...[New Thread 0x40ff4450 (LWP 2944)] [New Thread 0x411f4450 (LWP 2945)] [New Thread 0x413f4450 (LWP 2946)] done Creating alloc threads...[New Thread 0x415f4450 (LWP 2947)] [New Thread 0x417f4450 (LWP 2948)] [New Thread 0x419f4450 (LWP 2949)] done Creating lock threads...[New Thread 0x41bf4450 (LWP 2950)] [New Thread 0x41df4450 (LWP 2951)] [New Thread 0x41ff4450 (LWP 2952)] done running...printing oldest/median age/newest Program received signal SIG64, Real-time event 64. [Switching to Thread 0x41ff4450 (LWP 2952)] 0x4028f848 in pthread_cond_timedwait@@GLIBC_2.4 () from /lib/arm-linux-gnueabihf/libpthread.so.0 (gdb) handle SIG64 nostop noprint Signal Stop Print Pass to program Description SIG64 No No Yes Real-time event 64 (gdb) cont Continuing. . *** *** runtime error: *** An array subscript was out of range. *** file "../src/runtime/common/RTCollector.m3", line 418 *** Program received signal SIGABRT, Aborted. [Switching to Thread 0x40ff4450 (LWP 2944)] 0x402cfbfc in raise () from /lib/arm-linux-gnueabihf/libc.so.6 (gdb) where #0 0x402cfbfc in raise () from /lib/arm-linux-gnueabihf/libc.so.6 #1 0x402d397c in abort () from /lib/arm-linux-gnueabihf/libc.so.6 #2 0x00000000 in ?? () (gdb) It runs fine with user threads on AMD64_LINUX now. Haven't tried that on ARM_LINUX but would be surprised if it didn't work. Yes the pthreads seem "more reliable" than they used to be, but I would never use them in any sort of mission-critical application. I use Modula-3 programs for (among other things) financial transactions and real-time motor controllers... Mika From mika at async.caltech.edu Fri Jan 3 18:19:32 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Fri, 03 Jan 2014 09:19:32 -0800 Subject: [M3devel] AMD64_LINUX update... Message-ID: <20140103171932.F28D61A2080@async.async.caltech.edu> Hi again m3devel, With the head and user threads, after bootstrapping via AMD64_FREEBSD, I'm getting everything (apparently) working EXCEPT one thing: stubgen (and my derivative of it, the Scheme-stubgen "sstubgen") goes into an infinite loop, looks like it's happening on the stack because I run out of memory rather quickly, on certain types local to my own system. Has anyone seen this before? I've never seen it on any 32-bit system but I have a vague memory of seeing it on 64-bit systems. Will debug over the weekend... I'd be grateful for any input from the list on the pthreads issues, though. Mika From mika at async.caltech.edu Sat Jan 4 19:55:11 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sat, 04 Jan 2014 10:55:11 -0800 Subject: [M3devel] performance problems in CM3 Text operations on AMD64_LINUX Message-ID: <20140104185511.9000A1A2080@async.async.caltech.edu> I have this program that generates Modula-3 code from XML. It uses Wr.PutText a lot, on small strings. It runs 10 times faster under PM3 on FreeBSD/i386 on a 2 GHz old Pentium III type computer than under CM3 (at head) on Debian/amd64 on a 4 GHz Core i7 4960X. (176)rover:~/t/calarm/fix/gen/src>time ../FreeBSD4/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml 0.419u 0.089s 0:00.70 70.0% 471+27608k 0+546io 0pf+0w (29)truffles:~/t/calarm/fix/gen/src>time ../AMD64_LINUX/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml 1.952u 0.232s 0:05.40 40.3% 0+0k 0+11504io 0pf+0w I believe the issue is that Text.m3 is making a ton of system calls! Here's a typical traceback: (gdb) where #0 0x00002aaaaaacb770 in ?? () #1 0x00002aaaaaacba1b in gettimeofday () #2 0x00002aaaac468f8a in gettimeofday () at ../sysdeps/unix/sysv/linux/x86_64/gettimeofday.S:37 #3 0x000000000048c10c in TimePosix__Now () at ../src/time/POSIX/TimePosixC.c:50 #4 0x000000000048b4d2 in Time__Now () at ../src/time/POSIX/TimePosix.m3:14 #5 0x000000000049cbcf in TextStats__NoteFinished (M3_ESffkp_o=) at ../src/text/TextStats.m3:64 #6 0x00000000004921de in Text__SetChars (M3_CKMnXU_a=, M3_Bd56fi_t=, M3_Cwb5VA_start=) at ../src/text/Text.m3:512 #7 0x0000000000441a26 in UnsafeWr__FastPutText (M3_BxxOH1_wr=, M3_Bd56fi_t=) at ../src/rw/Wr.m3:100 #8 0x000000000044192f in Wr__PutText (M3_BxxOH1_wr=, M3_Bd56fi_t=) at ../src/rw/Wr.m3:89 #9 0x000000000041627f in M3Syntax__TextSetPutList (M3_CT76zs_s=, M3_BxxOH1_wr=, M3_AicXUJ_quoted=) at ../src/M3Syntax.m3:129 #10 0x00000000004169cf in M3Syntax__EnumToText (M3_Bqnwti_e=, M3_ACGgGY_env=) at ../src/M3Syntax.m3:173 #11 0x000000000041791a in M3Syntax__UnstrToText (M3_AIPC2K_u=, M3_ACGgGY_env=) at ../src/M3Syntax.m3:295 #12 0x0000000000414ad4 in M3File__DumpInterface (M3_CjySNq_t=, M3_Bd56fi_dirPath=, M3_AicXUJ_updateMakefile=) at ../src/M3File.m3:78 #13 0x000000000040f3f9 in Fields__Do__MakeFieldsFiles.1150 () at ../AMD64_LINUX/Fields.m3:526 #14 0x000000000040a713 in Fields__Do (M3_DsEykq_xml=, M3_Bd56fi_versionString=, M3_Bd56fi_beginStringV=, M3_D9m5ya_fieldTbl=, M3_Bd56fi_Dir=, M3_Bd56fi_SrcDir=, M3_AZx9O5_binaryFields=, M3_CebKnt_specialFields=, M3_CT76zs_checkedTypes=) at ../AMD64_LINUX/Fields.m3:538 #15 0x0000000000413948 in Main_M3 (M3_AcxOUs_mode=) at ../AMD64_LINUX/Main.m3:431 #16 0x00000000004742ad in RTLinker__RunMainBody (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:408 #17 0x0000000000473638 in RTLinker__AddUnitI (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:115 #18 0x00000000004736cc in RTLinker__AddUnit (M3_DjPxE5_b=) at ../src/runtime/common/RTLinker.m3:124 #19 0x0000000000405df8 in main (argc=8, argv=0x7fffffffe2b8, envp=0x7fffffffe300) at _m3main.c:22 (gdb) I believe all the TextStats operations are killing the performance. Can we remove them? Here is SetChars: PROCEDURE SetChars (VAR a: ARRAY OF CHAR; t: T; start: CARDINAL) = BEGIN TextStats.NoteGround (TextStats.Op.SetChars); TextStats.NoteGround (TextStats.Op.get_chars); t.get_chars (a, start); TextStats.NoteFinished (TextStats.Op.get_chars); TextStats.NoteFinished (TextStats.Op.SetChars); END SetChars; I'm not sure how best to do this. Removing them without killing functionality might not be easy. Looks like a case for conditional compilation. Maybe generate the .m3s from some other source, with and without the TextStats, depending on a setting in m3makefile? Second best would be to have a boolean flag in Text.m3 so the TextStats never get called. Mika From dragisha at m3w.org Sat Jan 4 22:04:22 2014 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 4 Jan 2014 22:04:22 +0100 Subject: [M3devel] AMD64_LINUX update... In-Reply-To: <20140103171932.F28D61A2080@async.async.caltech.edu> References: <20140103171932.F28D61A2080@async.async.caltech.edu> Message-ID: Hi Mika, I have this for LINUXLIBC6, and I am using it for remake of HEAD: cm3-bin-core-LINUXLIBC6-d5.9.0-i686-pc-linux-gnu-2013-11-20-19-55-28.tgz Can you build and share cm3-bin-core for AMD64_LINUX? TIA, dd On 03 Jan 2014, at 18:19, mika at async.caltech.edu wrote: > > Hi again m3devel, > > With the head and user threads, after bootstrapping via AMD64_FREEBSD, > I'm getting everything (apparently) working EXCEPT one thing: > > stubgen (and my derivative of it, the Scheme-stubgen "sstubgen") goes > into an infinite loop, looks like it's happening on the stack because I > run out of memory rather quickly, on certain types local to my own system. > > Has anyone seen this before? I've never seen it on any 32-bit system > but I have a vague memory of seeing it on 64-bit systems. > > Will debug over the weekend... > > I'd be grateful for any input from the list on the pthreads issues, though. > > Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Sat Jan 4 22:12:00 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sat, 04 Jan 2014 13:12:00 -0800 Subject: [M3devel] AMD64_LINUX update... In-Reply-To: References: <20140103171932.F28D61A2080@async.async.caltech.edu> Message-ID: <20140104211200.A80571A2080@async.async.caltech.edu> Sure... what do I do? Is there a script? Or do I tar up a selected set of directories? I can do this for both AMD64_LINUX (user threads) and ARM_LINUX (pthreads) I might still have AMD64_LINUX (pthreads) Also have AMD64_FREEBSD (pthreads) from the head. =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: > >--Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421 >Content-Type: multipart/alternative; > boundary="Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657" > > >--Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657 >Content-Transfer-Encoding: quoted-printable >Content-Type: text/plain; > charset=us-ascii > >Hi Mika, > >I have this for LINUXLIBC6, and I am using it for remake of HEAD: = >cm3-bin-core-LINUXLIBC6-d5.9.0-i686-pc-linux-gnu-2013-11-20-19-55-28.tgz > >Can you build and share cm3-bin-core for AMD64_LINUX? > >TIA, >dd > >On 03 Jan 2014, at 18:19, mika at async.caltech.edu wrote: > >>=20 >> Hi again m3devel, >>=20 >> With the head and user threads, after bootstrapping via AMD64_FREEBSD, >> I'm getting everything (apparently) working EXCEPT one thing: >>=20 >> stubgen (and my derivative of it, the Scheme-stubgen "sstubgen") goes >> into an infinite loop, looks like it's happening on the stack because = >I >> run out of memory rather quickly, on certain types local to my own = >system. >>=20 >> Has anyone seen this before? I've never seen it on any 32-bit system >> but I have a vague memory of seeing it on 64-bit systems. >>=20 >> Will debug over the weekend... >>=20 >> I'd be grateful for any input from the list on the pthreads issues, = >though. >>=20 >> Mika > > >--Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657 >Content-Transfer-Encoding: quoted-printable >Content-Type: text/html; > charset=us-ascii > >charset=3Dus-ascii">-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">Hi = >Mika,

I have this for LINUXLIBC6, and I am using it = >for remake of HEAD: color: rgb(255, 59, 29); font-family: Monaco; font-size: = >13px;">cm3-bin-core-LINUXLIBC6-d5.9.0-i686-pc-linux-gnu-2013-11-20-19-55-2= >8.tgz
color: rgb(255, 59, 29); font-family: Monaco; font-size: = >13px;">
orphans: 2; text-align: -webkit-auto; widows: 2;">Can you build and = >share cm3-bin-core for AMD64_LINUX?
apple-content-edited=3D"true">style=3D"border-collapse: separate; font-family: Candara; = >border-spacing: 0px;">style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: = >Helvetica; font-style: normal; font-variant: normal; font-weight: = >normal; letter-spacing: normal; line-height: normal; orphans: 2; = >text-align: -webkit-auto; text-indent: 0px; text-transform: none; = >white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; ">
break-word; -webkit-nbsp-mode: space; -webkit-line-break: = >after-white-space; ">style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: = >Helvetica; font-style: normal; font-variant: normal; font-weight: = >normal; letter-spacing: normal; line-height: normal; orphans: 2; = >text-align: -webkit-auto; text-indent: 0px; text-transform: none; = >white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; ">
break-word; -webkit-nbsp-mode: space; -webkit-line-break: = >after-white-space; ">
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; = >">TIA,
space; -webkit-line-break: after-white-space; ">dd
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; = >">

type=3D"cite">
Hi again m3devel,

With the head and user = >threads, after bootstrapping via AMD64_FREEBSD,
I'm getting = >everything (apparently) working EXCEPT one thing:

stubgen (and my = >derivative of it, the Scheme-stubgen "sstubgen") goes
into an = >infinite loop, looks like it's happening on the stack because I
run = >out of memory rather quickly, on certain types local to my own = >system.

Has anyone seen this before?  I've never seen it on = >any 32-bit system
but I have a vague memory of seeing it on 64-bit = >systems.

Will debug over the weekend...

I'd be grateful = >for any input from the list on the pthreads issues, though.

= >    Mika

>= > >--Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657-- > >--Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421 >Content-Transfer-Encoding: 7bit >Content-Disposition: attachment; > filename=signature.asc >Content-Type: application/pgp-signature; > name=signature.asc >Content-Description: Message signed with OpenPGP using GPGMail > >-----BEGIN PGP SIGNATURE----- >Comment: GPGTools - http://gpgtools.org > >iQEcBAEBAgAGBQJSyHdWAAoJEJtljYXUJo8x7G4H/i04W2KIer4qcBJO0uVLaI2+ >taHLJ9ewgRyfrd3qfp7YkgmoidxVMy6XMRIuwFpNBjz8YE5YK5TU9Awpmr4UKVvM >zGa9eeWWMg+SX6WmiKnIO4WxiF2oNkp3mybuX4oOjjqUN4nhW0iGtycgIlGkDaR7 >0CL5I4D0P8bSkhunE4iATX9I/k+WVFghZhRWr8BULIHgRlD1xG6NZ0EhCcJk1mY2 >hQBKgJEdpiBUfG84cbu0uE5c3hrposmbjuFFaegG0Q1Pr+zZAnbV5C08RhLawyHn >1VPwWGF1R/d1KgVBr4l19KxNi4aACJtov5ODGXXJbfOFyPi25WF6A4SbjP1aJQg= >=sMKJ >-----END PGP SIGNATURE----- > >--Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421-- From mika at async.caltech.edu Sat Jan 4 22:15:51 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sat, 04 Jan 2014 13:15:51 -0800 Subject: [M3devel] thread tester results on ARM_LINUX In-Reply-To: References: <20140103161343.F25291A2080@async.async.caltech.edu> Message-ID: <20140104211551.54F481A2080@async.async.caltech.edu> Antony Hosking writes: >I wonder of the redzone size for AMD 64 stack frames is not set properly.=20= > Tony, Are you talking about the difference between the ARM and AMD64 results? The AMD64 is also a machine with 12 logical (HT) processors whereas the ARM is a single processor. That could make a difference in how it fails. By the way, would you know: The following output on the amd64: Continuing. ....pthread_mutex_destroy:EBUSY ...pthread_mutex_destroy:EBUSY Is that an assertion failing in the pthreads library? My program isn't printing this stuff: it's coming from ThreadPThread or some such code. Mika > >Sent from my iPhone > >> On Jan 3, 2014, at 10:13 AM, mika at async.caltech.edu wrote: >>=20 >> (78)raspberrypi:~/cm3-cvs-anon/cm3/m3-libs/m3core/tests/thread>gdb ARM_LIN= >UX/threadtest >> GNU gdb (GDB) 7.4.1-debian >> Copyright (C) 2012 Free Software Foundation, Inc. >> License GPLv3+: GNU GPL version 3 or later ml> >> This is free software: you are free to change and redistribute it. >> There is NO WARRANTY, to the extent permitted by law. Type "show copying"= > >> and "show warranty" for details. >> This GDB was configured as "arm-linux-gnueabihf". >> For bug reporting instructions, please see: >> ... >> Reading symbols from /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/th= >read/ARM_LINUX/threadtest...done. >> (gdb) handle SIGILL=20 >> Signal Stop Print Pass to program Description >> SIGILL Yes Yes Yes Illegal instruction >> (gdb) handle SIGILL nostop noprint >> Signal Stop Print Pass to program Description >> SIGILL No No Yes Illegal instruction >> (gdb) run >> Starting program: /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/threa= >d/ARM_LINUX/threadtest=20 >> [Thread debugging using libthread_db enabled] >> Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.= >1". >> Writing file...done >> Creating read threads...[New Thread 0x409f4450 (LWP 2941)] >> [New Thread 0x40bf4450 (LWP 2942)] >> [New Thread 0x40df4450 (LWP 2943)] >> done >> Creating fork threads...[New Thread 0x40ff4450 (LWP 2944)] >> [New Thread 0x411f4450 (LWP 2945)] >> [New Thread 0x413f4450 (LWP 2946)] >> done >> Creating alloc threads...[New Thread 0x415f4450 (LWP 2947)] >> [New Thread 0x417f4450 (LWP 2948)] >> [New Thread 0x419f4450 (LWP 2949)] >> done >> Creating lock threads...[New Thread 0x41bf4450 (LWP 2950)] >> [New Thread 0x41df4450 (LWP 2951)] >> [New Thread 0x41ff4450 (LWP 2952)] >> done >> running...printing oldest/median age/newest >>=20 >> Program received signal SIG64, Real-time event 64. >> [Switching to Thread 0x41ff4450 (LWP 2952)] >> 0x4028f848 in pthread_cond_timedwait@@GLIBC_2.4 () from /lib/arm-linux-gnu= >eabihf/libpthread.so.0 >> (gdb) handle SIG64 nostop noprint >> Signal Stop Print Pass to program Description >> SIG64 No No Yes Real-time event 64 >> (gdb) cont >> Continuing. >> . >>=20 >> *** >> *** runtime error: >> *** An array subscript was out of range. >> *** file "../src/runtime/common/RTCollector.m3", line 418 >> *** >>=20 >>=20 >> Program received signal SIGABRT, Aborted. >> [Switching to Thread 0x40ff4450 (LWP 2944)] >> 0x402cfbfc in raise () from /lib/arm-linux-gnueabihf/libc.so.6 >> (gdb) where >> #0 0x402cfbfc in raise () from /lib/arm-linux-gnueabihf/libc.so.6 >> #1 0x402d397c in abort () from /lib/arm-linux-gnueabihf/libc.so.6 >> #2 0x00000000 in ?? () >> (gdb)=20 >>=20 >> It runs fine with user threads on AMD64_LINUX now. Haven't tried that on >> ARM_LINUX but would be surprised if it didn't work. >>=20 >> Yes the pthreads seem "more reliable" than they used to be, but I would >> never use them in any sort of mission-critical application. I use >> Modula-3 programs for (among other things) financial transactions and >> real-time motor controllers... >>=20 >> Mika From dragisha at m3w.org Sat Jan 4 22:52:32 2014 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 4 Jan 2014 22:52:32 +0100 Subject: [M3devel] AMD64_LINUX update... In-Reply-To: <20140104211200.A80571A2080@async.async.caltech.edu> References: <20140103171932.F28D61A2080@async.async.caltech.edu> <20140104211200.A80571A2080@async.async.caltech.edu> Message-ID: -rwxrwxr-x. 1 root 969 Feb 1 2010 scripts/do-cm3-all.sh* -rwxrwxr-x. 1 root 985 Feb 1 2010 scripts/do-cm3-base.sh* -rwxrwxr-x. 1 root 994 Feb 1 2010 scripts/do-cm3-caltech-parser.sh* -rwxrwxr-x. 1 root 998 Feb 1 2010 scripts/do-cm3-coll.sh* -rwxrwxr-x. 1 root 984 Feb 1 2010 scripts/do-cm3-comm.sh* -rwxrwxr-x. 1 root 985 Feb 1 2010 scripts/do-cm3-core.sh* -rwxrwxr-x. 1 root 1072 Feb 1 2010 scripts/do-cm3-front.sh* -rwxrwxr-x. 1 root 984 Feb 1 2010 scripts/do-cm3-gui.sh* -rwxrwxr-x. 1 root 926 Jun 7 2009 scripts/do-cm3-m3gdb.sh* -rwxrwxr-x. 1 root 984 Feb 1 2010 scripts/do-cm3-min.sh* -rwxrwxr-x. 1 root 969 Nov 11 2010 scripts/do-cm3-std.sh* -rwxrwxr-x. 1 root 938 Feb 6 2013 scripts/do-pkg.sh* there they are. I don?t do this too often, but both do-cm3-core and do-cm3-min will do for what I need. TIA, dd On 04 Jan 2014, at 22:12, mika at async.caltech.edu wrote: > Sure... what do I do? Is there a script? Or do I tar up a selected set of directories? > > I can do this for both AMD64_LINUX (user threads) > and ARM_LINUX (pthreads) > > I might still have AMD64_LINUX (pthreads) > > Also have AMD64_FREEBSD (pthreads) from the head. > > =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: >> >> --Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421 >> Content-Type: multipart/alternative; >> boundary="Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657" >> >> >> --Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657 >> Content-Transfer-Encoding: quoted-printable >> Content-Type: text/plain; >> charset=us-ascii >> >> Hi Mika, >> >> I have this for LINUXLIBC6, and I am using it for remake of HEAD: = >> cm3-bin-core-LINUXLIBC6-d5.9.0-i686-pc-linux-gnu-2013-11-20-19-55-28.tgz >> >> Can you build and share cm3-bin-core for AMD64_LINUX? >> >> TIA, >> dd >> >> On 03 Jan 2014, at 18:19, mika at async.caltech.edu wrote: >> >>> =20 >>> Hi again m3devel, >>> =20 >>> With the head and user threads, after bootstrapping via AMD64_FREEBSD, >>> I'm getting everything (apparently) working EXCEPT one thing: >>> =20 >>> stubgen (and my derivative of it, the Scheme-stubgen "sstubgen") goes >>> into an infinite loop, looks like it's happening on the stack because = >> I >>> run out of memory rather quickly, on certain types local to my own = >> system. >>> =20 >>> Has anyone seen this before? I've never seen it on any 32-bit system >>> but I have a vague memory of seeing it on 64-bit systems. >>> =20 >>> Will debug over the weekend... >>> =20 >>> I'd be grateful for any input from the list on the pthreads issues, = >> though. >>> =20 >>> Mika >> >> >> --Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657 >> Content-Transfer-Encoding: quoted-printable >> Content-Type: text/html; >> charset=us-ascii >> >> > charset=3Dus-ascii">> -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">Hi = >> Mika,

I have this for LINUXLIBC6, and I am using it = >> for remake of HEAD: > color: rgb(255, 59, 29); font-family: Monaco; font-size: = >> 13px;">cm3-bin-core-LINUXLIBC6-d5.9.0-i686-pc-linux-gnu-2013-11-20-19-55-2= >> 8.tgz
> color: rgb(255, 59, 29); font-family: Monaco; font-size: = >> 13px;">
> orphans: 2; text-align: -webkit-auto; widows: 2;">Can you build and = >> share cm3-bin-core for AMD64_LINUX?
> apple-content-edited=3D"true">> style=3D"border-collapse: separate; font-family: Candara; = >> border-spacing: 0px;">> style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: = >> Helvetica; font-style: normal; font-variant: normal; font-weight: = >> normal; letter-spacing: normal; line-height: normal; orphans: 2; = >> text-align: -webkit-auto; text-indent: 0px; text-transform: none; = >> white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; ">
> break-word; -webkit-nbsp-mode: space; -webkit-line-break: = >> after-white-space; ">> style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: = >> Helvetica; font-style: normal; font-variant: normal; font-weight: = >> normal; letter-spacing: normal; line-height: normal; orphans: 2; = >> text-align: -webkit-auto; text-indent: 0px; text-transform: none; = >> white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; ">
> break-word; -webkit-nbsp-mode: space; -webkit-line-break: = >> after-white-space; ">
> -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; = >> ">TIA,
> space; -webkit-line-break: after-white-space; ">dd
> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >> -webkit-line-break: after-white-space; = >> ">

> type=3D"cite">
Hi again m3devel,

With the head and user = >> threads, after bootstrapping via AMD64_FREEBSD,
I'm getting = >> everything (apparently) working EXCEPT one thing:

stubgen (and my = >> derivative of it, the Scheme-stubgen "sstubgen") goes
into an = >> infinite loop, looks like it's happening on the stack because I
run = >> out of memory rather quickly, on certain types local to my own = >> system.

Has anyone seen this before?  I've never seen it on = >> any 32-bit system
but I have a vague memory of seeing it on 64-bit = >> systems.

Will debug over the weekend...

I'd be grateful = >> for any input from the list on the pthreads issues, though.

= >>     Mika

>> = >> >> --Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657-- >> >> --Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421 >> Content-Transfer-Encoding: 7bit >> Content-Disposition: attachment; >> filename=signature.asc >> Content-Type: application/pgp-signature; >> name=signature.asc >> Content-Description: Message signed with OpenPGP using GPGMail >> >> -----BEGIN PGP SIGNATURE----- >> Comment: GPGTools - http://gpgtools.org >> >> iQEcBAEBAgAGBQJSyHdWAAoJEJtljYXUJo8x7G4H/i04W2KIer4qcBJO0uVLaI2+ >> taHLJ9ewgRyfrd3qfp7YkgmoidxVMy6XMRIuwFpNBjz8YE5YK5TU9Awpmr4UKVvM >> zGa9eeWWMg+SX6WmiKnIO4WxiF2oNkp3mybuX4oOjjqUN4nhW0iGtycgIlGkDaR7 >> 0CL5I4D0P8bSkhunE4iATX9I/k+WVFghZhRWr8BULIHgRlD1xG6NZ0EhCcJk1mY2 >> hQBKgJEdpiBUfG84cbu0uE5c3hrposmbjuFFaegG0Q1Pr+zZAnbV5C08RhLawyHn >> 1VPwWGF1R/d1KgVBr4l19KxNi4aACJtov5ODGXXJbfOFyPi25WF6A4SbjP1aJQg= >> =sMKJ >> -----END PGP SIGNATURE----- >> >> --Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421-- -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Sat Jan 4 22:58:53 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sat, 04 Jan 2014 13:58:53 -0800 Subject: [M3devel] AMD64_LINUX update... In-Reply-To: References: <20140103171932.F28D61A2080@async.async.caltech.edu> <20140104211200.A80571A2080@async.async.caltech.edu> Message-ID: <20140104215853.09F0D1A2080@async.async.caltech.edu> Hi dd, I already have everything built. What I'm asking is how I go from what I have to a tar file. What is included? Is there a script for that? I can just give you the full binary directory or the full cm3 system with sources, but I suspect there is a reduced set you would prefer? BTW (not sure if this is relevant) but Jay's ./boot1.py and ./boot2.sh scripts in python work fine. If you already have cm3 from the head it ought to be possible to cross-build. But I did have problems going from 32 to 64 bits... Mika =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: > >--Apple-Mail=_F0511A87-0318-4B01-8056-ED3EF8213304 >Content-Type: multipart/alternative; > boundary="Apple-Mail=_67A3EF09-2BA4-4414-BDCB-1E89671A1F12" > > >--Apple-Mail=_67A3EF09-2BA4-4414-BDCB-1E89671A1F12 >Content-Transfer-Encoding: quoted-printable >Content-Type: text/plain; > charset=windows-1252 > >-rwxrwxr-x. 1 root 969 Feb 1 2010 scripts/do-cm3-all.sh* >-rwxrwxr-x. 1 root 985 Feb 1 2010 scripts/do-cm3-base.sh* >-rwxrwxr-x. 1 root 994 Feb 1 2010 scripts/do-cm3-caltech-parser.sh* >-rwxrwxr-x. 1 root 998 Feb 1 2010 scripts/do-cm3-coll.sh* >-rwxrwxr-x. 1 root 984 Feb 1 2010 scripts/do-cm3-comm.sh* >-rwxrwxr-x. 1 root 985 Feb 1 2010 scripts/do-cm3-core.sh* >-rwxrwxr-x. 1 root 1072 Feb 1 2010 scripts/do-cm3-front.sh* >-rwxrwxr-x. 1 root 984 Feb 1 2010 scripts/do-cm3-gui.sh* >-rwxrwxr-x. 1 root 926 Jun 7 2009 scripts/do-cm3-m3gdb.sh* >-rwxrwxr-x. 1 root 984 Feb 1 2010 scripts/do-cm3-min.sh* >-rwxrwxr-x. 1 root 969 Nov 11 2010 scripts/do-cm3-std.sh* >-rwxrwxr-x. 1 root 938 Feb 6 2013 scripts/do-pkg.sh* > >there they are. I don=92t do this too often, but both do-cm3-core and = >do-cm3-min will do for what I need. > >TIA, >dd From dragisha at m3w.org Sat Jan 4 23:10:04 2014 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 4 Jan 2014 23:10:04 +0100 Subject: [M3devel] AMD64_LINUX update... In-Reply-To: <20140104215853.09F0D1A2080@async.async.caltech.edu> References: <20140103171932.F28D61A2080@async.async.caltech.edu> <20140104211200.A80571A2080@async.async.caltech.edu> <20140104215853.09F0D1A2080@async.async.caltech.edu> Message-ID: <7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org> do-cm3-bin will build a tar.gz, I think. And it is not too big. I did not try boot*.py as this box I would like to build for is remote and all? I have src.rpm and I only need this cm3-min? to make it work and build itself on remote box. TIA, dd On 04 Jan 2014, at 22:58, mika at async.caltech.edu wrote: > Hi dd, > > I already have everything built. > > What I'm asking is how I go from what I have to a tar file. What is > included? Is there a script for that? > > I can just give you the full binary directory or the full cm3 system with > sources, but I suspect there is a reduced set you would prefer? > > BTW (not sure if this is relevant) but Jay's ./boot1.py and ./boot2.sh > scripts in python work fine. If you already have cm3 from the head > it ought to be possible to cross-build. But I did have problems going from > 32 to 64 bits... > > Mika > > =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: >> >> --Apple-Mail=_F0511A87-0318-4B01-8056-ED3EF8213304 >> Content-Type: multipart/alternative; >> boundary="Apple-Mail=_67A3EF09-2BA4-4414-BDCB-1E89671A1F12" >> >> >> --Apple-Mail=_67A3EF09-2BA4-4414-BDCB-1E89671A1F12 >> Content-Transfer-Encoding: quoted-printable >> Content-Type: text/plain; >> charset=windows-1252 >> >> -rwxrwxr-x. 1 root 969 Feb 1 2010 scripts/do-cm3-all.sh* >> -rwxrwxr-x. 1 root 985 Feb 1 2010 scripts/do-cm3-base.sh* >> -rwxrwxr-x. 1 root 994 Feb 1 2010 scripts/do-cm3-caltech-parser.sh* >> -rwxrwxr-x. 1 root 998 Feb 1 2010 scripts/do-cm3-coll.sh* >> -rwxrwxr-x. 1 root 984 Feb 1 2010 scripts/do-cm3-comm.sh* >> -rwxrwxr-x. 1 root 985 Feb 1 2010 scripts/do-cm3-core.sh* >> -rwxrwxr-x. 1 root 1072 Feb 1 2010 scripts/do-cm3-front.sh* >> -rwxrwxr-x. 1 root 984 Feb 1 2010 scripts/do-cm3-gui.sh* >> -rwxrwxr-x. 1 root 926 Jun 7 2009 scripts/do-cm3-m3gdb.sh* >> -rwxrwxr-x. 1 root 984 Feb 1 2010 scripts/do-cm3-min.sh* >> -rwxrwxr-x. 1 root 969 Nov 11 2010 scripts/do-cm3-std.sh* >> -rwxrwxr-x. 1 root 938 Feb 6 2013 scripts/do-pkg.sh* >> >> there they are. I don=92t do this too often, but both do-cm3-core and = >> do-cm3-min will do for what I need. >> >> TIA, >> dd -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Sat Jan 4 23:23:21 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sat, 04 Jan 2014 14:23:21 -0800 Subject: [M3devel] can't write to repository Message-ID: <20140104222321.B53051A2080@async.async.caltech.edu> Hi m3devel, I can't seem to write to the CVS repository. I get: (62)rover:~/cm3-writable/cm3/scripts>cvs up -d . mika at birch.elegosoft.com's password: Has my account been removed? I don't think I've changed keys. In any case, could someone please commit the following change: ~/cm3-cvs-anon/cm3/scripts>cvs diff sysinfo.sh Index: sysinfo.sh =================================================================== RCS file: /usr/cvs/cm3/scripts/sysinfo.sh,v retrieving revision 1.109 diff -r1.109 sysinfo.sh 273a274 > arm*) CM3_TARGET=ARM_LINUX;; Mika From mika at async.caltech.edu Sat Jan 4 23:48:13 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sat, 04 Jan 2014 14:48:13 -0800 Subject: [M3devel] AMD64_LINUX update... In-Reply-To: <7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org> References: <20140103171932.F28D61A2080@async.async.caltech.edu> <20140104211200.A80571A2080@async.async.caltech.edu> <20140104215853.09F0D1A2080@async.async.caltech.edu> <7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org> Message-ID: <20140104224813.34D171A2080@async.async.caltech.edu> I think the script you meant was make-bin-dist-min.sh The results are at: http://rover.gcapltd.com/~mika/cm3-bin-core-AMD64_LINUX-d5.9.0-x86_64-unknown-linux-gnu-2014-01-04-22-18-35.tgz Please upload this to the CM3 website if it looks OK to you and you know how to upload it. It's set up to use user threads. Mika From dragisha at m3w.org Sat Jan 4 23:57:55 2014 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 4 Jan 2014 23:57:55 +0100 Subject: [M3devel] AMD64_LINUX update... In-Reply-To: <20140104224813.34D171A2080@async.async.caltech.edu> References: <20140103171932.F28D61A2080@async.async.caltech.edu> <20140104211200.A80571A2080@async.async.caltech.edu> <20140104215853.09F0D1A2080@async.async.caltech.edu> <7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org> <20140104224813.34D171A2080@async.async.caltech.edu> Message-ID: <833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org> I am using it to bootstrap my rpm, did you try copying to: /var/www/modula3.elegosoft.com/cm3/uploaded-archives/ on birch? I will make my rpm tomorrow, and try uploading all of it then. Thanks, dd On 04 Jan 2014, at 23:48, mika at async.caltech.edu wrote: > > I think the script you meant was make-bin-dist-min.sh > > The results are at: > > http://rover.gcapltd.com/~mika/cm3-bin-core-AMD64_LINUX-d5.9.0-x86_64-unknown-linux-gnu-2014-01-04-22-18-35.tgz > > Please upload this to the CM3 website if it looks OK to you and you know how to upload it. > > It's set up to use user threads. > > Mika > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Sat Jan 4 23:59:24 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sat, 04 Jan 2014 14:59:24 -0800 Subject: [M3devel] AMD64_LINUX update... In-Reply-To: <833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org> References: <20140103171932.F28D61A2080@async.async.caltech.edu> <20140104211200.A80571A2080@async.async.caltech.edu> <20140104215853.09F0D1A2080@async.async.caltech.edu> <7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org> <20140104224813.34D171A2080@async.async.caltech.edu> <833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org> Message-ID: <20140104225924.858A21A2080@async.async.caltech.edu> >I am using it to bootstrap my rpm, did you try copying to: = >/var/www/modula3.elegosoft.com/cm3/uploaded-archives/ on birch? Per my email of a few minutes ago, I am unable to get birch to accept my login credentials... From dragisha at m3w.org Sun Jan 5 01:44:17 2014 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 5 Jan 2014 01:44:17 +0100 Subject: [M3devel] AMD64_LINUX update... In-Reply-To: <20140104225924.858A21A2080@async.async.caltech.edu> References: <20140103171932.F28D61A2080@async.async.caltech.edu> <20140104211200.A80571A2080@async.async.caltech.edu> <20140104215853.09F0D1A2080@async.async.caltech.edu> <7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org> <20140104224813.34D171A2080@async.async.caltech.edu> <833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org> <20140104225924.858A21A2080@async.async.caltech.edu> Message-ID: <9CF45913-50ED-45F4-9774-0D0152016105@m3w.org> Copied, and did ?a sensible thing?, ran update script? It did not create as complete index.html as it was before. I will look into it tomorrow, it is too late for me now to believe my further fixing would not break more things :). On 04 Jan 2014, at 23:59, mika at async.caltech.edu wrote: >> I am using it to bootstrap my rpm, did you try copying to: = >> /var/www/modula3.elegosoft.com/cm3/uploaded-archives/ on birch? > > Per my email of a few minutes ago, I am unable to get birch to accept > my login credentials... -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Sun Jan 5 03:50:44 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sat, 04 Jan 2014 18:50:44 -0800 Subject: [M3devel] AMD64_LINUX update... In-Reply-To: <9CF45913-50ED-45F4-9774-0D0152016105@m3w.org> References: <20140103171932.F28D61A2080@async.async.caltech.edu> <20140104211200.A80571A2080@async.async.caltech.edu> <20140104215853.09F0D1A2080@async.async.caltech.edu> <7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org> <20140104224813.34D171A2080@async.async.caltech.edu> <833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org> <20140104225924.858A21A2080@async.async.caltech.edu> <9CF45913-50ED-45F4-9774-0D0152016105@m3w.org> Message-ID: <20140105025044.9FC551A2080@async.async.caltech.edu> Another favor? Could you upload to CM3 the file at http://rover.gcapltd.com/~mika/cm3-bin-core-ARM_LINUX-d5.9.0-armv6l-unknown-linux-gnueabi-2014-01-04-22-23-51.tgz If there is a "notes" section, you can note that the distribution works on 1. Raspberry Pi running Raspbian Wheezy Linux raspberrypi 3.6.11+ #538 PREEMPT Fri Aug 30 20:42:08 BST 2013 armv6l 2. BeagleBone Black running (BeagleBoard)Ubuntu 13.10 (GNU/Linux 3.8.13-bone28 armv7l) Linux arm 3.8.13-bone28 #1 SMP Fri Sep 13 01:11:14 UTC 2013 armv7l armv7l armv7l GNU/Linux (Anyone reading this mail who wants a copy can of course also use the link to rover above.) Mika From dragisha at m3w.org Sun Jan 5 21:15:23 2014 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 5 Jan 2014 21:15:23 +0100 Subject: [M3devel] AMD64_LINUX update... In-Reply-To: <20140105025044.9FC551A2080@async.async.caltech.edu> References: <20140103171932.F28D61A2080@async.async.caltech.edu> <20140104211200.A80571A2080@async.async.caltech.edu> <20140104215853.09F0D1A2080@async.async.caltech.edu> <7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org> <20140104224813.34D171A2080@async.async.caltech.edu> <833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org> <20140104225924.858A21A2080@async.async.caltech.edu> <9CF45913-50ED-45F4-9774-0D0152016105@m3w.org> <20140105025044.9FC551A2080@async.async.caltech.edu> Message-ID: <6F081BEB-ED1C-481F-9389-C36546AAF2D6@m3w.org> Also on http://opencm3.net/snaps/ On 05 Jan 2014, at 03:50, mika at async.caltech.edu wrote: > http://rover.gcapltd.com/~mika/cm3-bin-core-ARM_LINUX-d5.9.0-armv6l-unknown-linux-gnueabi-2014-01-04-22-23-51.tgz -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From dragisha at m3w.org Mon Jan 6 06:02:36 2014 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Mon, 6 Jan 2014 06:02:36 +0100 Subject: [M3devel] AMD64_LINUX update... In-Reply-To: <6F081BEB-ED1C-481F-9389-C36546AAF2D6@m3w.org> References: <20140103171932.F28D61A2080@async.async.caltech.edu> <20140104211200.A80571A2080@async.async.caltech.edu> <20140104215853.09F0D1A2080@async.async.caltech.edu> <7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org> <20140104224813.34D171A2080@async.async.caltech.edu> <833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org> <20140104225924.858A21A2080@async.async.caltech.edu> <9CF45913-50ED-45F4-9774-0D0152016105@m3w.org> <20140105025044.9FC551A2080@async.async.caltech.edu> <6F081BEB-ED1C-481F-9389-C36546AAF2D6@m3w.org> Message-ID: <9C223752-0979-4BC0-84C7-C8F83B9BF01B@m3w.org> There is hourly run script for user m3 which updates index file in this folder. It keeps deleteng your AMD_LINUX snapshot, apparently after indexing it. ARM_LINUX one is indexed and left in peace. I will look into this issue in few days. dd On 05 Jan 2014, at 21:15, Dragi?a Duri? wrote: > Also on http://opencm3.net/snaps/ > > > On 05 Jan 2014, at 03:50, mika at async.caltech.edu wrote: > >> http://rover.gcapltd.com/~mika/cm3-bin-core-ARM_LINUX-d5.9.0-armv6l-unknown-linux-gnueabi-2014-01-04-22-23-51.tgz > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From rodney_bates at lcwb.coop Mon Jan 6 20:41:53 2014 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 06 Jan 2014 13:41:53 -0600 Subject: [M3devel] performance problems in CM3 Text operations on AMD64_LINUX In-Reply-To: <20140104185511.9000A1A2080@async.async.caltech.edu> References: <20140104185511.9000A1A2080@async.async.caltech.edu> Message-ID: <52CB0701.9030307@lcwb.coop> On 01/04/2014 12:55 PM, mika at async.caltech.edu wrote: > I have this program that generates Modula-3 code from XML. > > It uses Wr.PutText a lot, on small strings. > > It runs 10 times faster under PM3 on FreeBSD/i386 on a 2 GHz old Pentium > III type computer than under CM3 (at head) on Debian/amd64 on a 4 GHz > Core i7 4960X. > > (176)rover:~/t/calarm/fix/gen/src>time ../FreeBSD4/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml > 0.419u 0.089s 0:00.70 70.0% 471+27608k 0+546io 0pf+0w > > (29)truffles:~/t/calarm/fix/gen/src>time ../AMD64_LINUX/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml > 1.952u 0.232s 0:05.40 40.3% 0+0k 0+11504io 0pf+0w > > I believe the issue is that Text.m3 is making a ton of system calls! > > Here's a typical traceback: > > (gdb) where > #0 0x00002aaaaaacb770 in ?? () > #1 0x00002aaaaaacba1b in gettimeofday () > #2 0x00002aaaac468f8a in gettimeofday () at ../sysdeps/unix/sysv/linux/x86_64/gettimeofday.S:37 > #3 0x000000000048c10c in TimePosix__Now () at ../src/time/POSIX/TimePosixC.c:50 > #4 0x000000000048b4d2 in Time__Now () at ../src/time/POSIX/TimePosix.m3:14 > #5 0x000000000049cbcf in TextStats__NoteFinished (M3_ESffkp_o=) at ../src/text/TextStats.m3:64 > #6 0x00000000004921de in Text__SetChars (M3_CKMnXU_a=, M3_Bd56fi_t=, > M3_Cwb5VA_start=) at ../src/text/Text.m3:512 > #7 0x0000000000441a26 in UnsafeWr__FastPutText (M3_BxxOH1_wr=, M3_Bd56fi_t=) at ../src/rw/Wr.m3:100 > #8 0x000000000044192f in Wr__PutText (M3_BxxOH1_wr=, M3_Bd56fi_t=) at ../src/rw/Wr.m3:89 > #9 0x000000000041627f in M3Syntax__TextSetPutList (M3_CT76zs_s=, M3_BxxOH1_wr=, > M3_AicXUJ_quoted=) at ../src/M3Syntax.m3:129 > #10 0x00000000004169cf in M3Syntax__EnumToText (M3_Bqnwti_e=, M3_ACGgGY_env=) at ../src/M3Syntax.m3:173 > #11 0x000000000041791a in M3Syntax__UnstrToText (M3_AIPC2K_u=, M3_ACGgGY_env=) at ../src/M3Syntax.m3:295 > #12 0x0000000000414ad4 in M3File__DumpInterface (M3_CjySNq_t=, M3_Bd56fi_dirPath=, > M3_AicXUJ_updateMakefile=) at ../src/M3File.m3:78 > #13 0x000000000040f3f9 in Fields__Do__MakeFieldsFiles.1150 () at ../AMD64_LINUX/Fields.m3:526 > #14 0x000000000040a713 in Fields__Do (M3_DsEykq_xml=, M3_Bd56fi_versionString=, > M3_Bd56fi_beginStringV=, M3_D9m5ya_fieldTbl=, M3_Bd56fi_Dir=, > M3_Bd56fi_SrcDir=, M3_AZx9O5_binaryFields=, M3_CebKnt_specialFields=, > M3_CT76zs_checkedTypes=) at ../AMD64_LINUX/Fields.m3:538 > #15 0x0000000000413948 in Main_M3 (M3_AcxOUs_mode=) at ../AMD64_LINUX/Main.m3:431 > #16 0x00000000004742ad in RTLinker__RunMainBody (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:408 > #17 0x0000000000473638 in RTLinker__AddUnitI (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:115 > #18 0x00000000004736cc in RTLinker__AddUnit (M3_DjPxE5_b=) at ../src/runtime/common/RTLinker.m3:124 > #19 0x0000000000405df8 in main (argc=8, argv=0x7fffffffe2b8, envp=0x7fffffffe300) at _m3main.c:22 > (gdb) > > I believe all the TextStats operations are killing the performance. Can we remove them? > > Here is SetChars: > > PROCEDURE SetChars (VAR a: ARRAY OF CHAR; t: T; start: CARDINAL) = > BEGIN > TextStats.NoteGround (TextStats.Op.SetChars); > TextStats.NoteGround (TextStats.Op.get_chars); > t.get_chars (a, start); > TextStats.NoteFinished (TextStats.Op.get_chars); > TextStats.NoteFinished (TextStats.Op.SetChars); > END SetChars; > > I'm not sure how best to do this. Removing them without killing functionality might not be > easy. Looks like a case for conditional compilation. > > Maybe generate the .m3s from some other source, with and without the TextStats, depending on > a setting in m3makefile? > > Second best would be to have a boolean flag in Text.m3 so the TextStats never get called. Try the fix I just checked in to TextClass.m3. It was already set up with such a boolean flag (TextClass.CollectStats, set FALSE), but in one of the instrumentation procedures, two system calls were outside the test thereof. Hopefully, this will give a good bit of relief for now, with a simple fix. > > Mika > > From rodney_bates at lcwb.coop Thu Jan 9 22:01:55 2014 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 09 Jan 2014 15:01:55 -0600 Subject: [M3devel] performance problems in CM3 Text operations on AMD64_LINUX In-Reply-To: <20140104185511.9000A1A2080@async.async.caltech.edu> References: <20140104185511.9000A1A2080@async.async.caltech.edu> Message-ID: <52CF0E43.2030701@lcwb.coop> On 01/04/2014 12:55 PM, mika at async.caltech.edu wrote: > I have this program that generates Modula-3 code from XML. > > It uses Wr.PutText a lot, on small strings. > > It runs 10 times faster under PM3 on FreeBSD/i386 on a 2 GHz old Pentium > III type computer than under CM3 (at head) on Debian/amd64 on a 4 GHz > Core i7 4960X. > > (176)rover:~/t/calarm/fix/gen/src>time ../FreeBSD4/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml > 0.419u 0.089s 0:00.70 70.0% 471+27608k 0+546io 0pf+0w > > (29)truffles:~/t/calarm/fix/gen/src>time ../AMD64_LINUX/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml > 1.952u 0.232s 0:05.40 40.3% 0+0k 0+11504io 0pf+0w > > I believe the issue is that Text.m3 is making a ton of system calls! > > Here's a typical traceback: > > (gdb) where > #0 0x00002aaaaaacb770 in ?? () > #1 0x00002aaaaaacba1b in gettimeofday () > #2 0x00002aaaac468f8a in gettimeofday () at ../sysdeps/unix/sysv/linux/x86_64/gettimeofday.S:37 > #3 0x000000000048c10c in TimePosix__Now () at ../src/time/POSIX/TimePosixC.c:50 > #4 0x000000000048b4d2 in Time__Now () at ../src/time/POSIX/TimePosix.m3:14 > #5 0x000000000049cbcf in TextStats__NoteFinished (M3_ESffkp_o=) at ../src/text/TextStats.m3:64 > #6 0x00000000004921de in Text__SetChars (M3_CKMnXU_a=, M3_Bd56fi_t=, > M3_Cwb5VA_start=) at ../src/text/Text.m3:512 > #7 0x0000000000441a26 in UnsafeWr__FastPutText (M3_BxxOH1_wr=, M3_Bd56fi_t=) at ../src/rw/Wr.m3:100 > #8 0x000000000044192f in Wr__PutText (M3_BxxOH1_wr=, M3_Bd56fi_t=) at ../src/rw/Wr.m3:89 > #9 0x000000000041627f in M3Syntax__TextSetPutList (M3_CT76zs_s=, M3_BxxOH1_wr=, > M3_AicXUJ_quoted=) at ../src/M3Syntax.m3:129 > #10 0x00000000004169cf in M3Syntax__EnumToText (M3_Bqnwti_e=, M3_ACGgGY_env=) at ../src/M3Syntax.m3:173 > #11 0x000000000041791a in M3Syntax__UnstrToText (M3_AIPC2K_u=, M3_ACGgGY_env=) at ../src/M3Syntax.m3:295 > #12 0x0000000000414ad4 in M3File__DumpInterface (M3_CjySNq_t=, M3_Bd56fi_dirPath=, > M3_AicXUJ_updateMakefile=) at ../src/M3File.m3:78 > #13 0x000000000040f3f9 in Fields__Do__MakeFieldsFiles.1150 () at ../AMD64_LINUX/Fields.m3:526 > #14 0x000000000040a713 in Fields__Do (M3_DsEykq_xml=, M3_Bd56fi_versionString=, > M3_Bd56fi_beginStringV=, M3_D9m5ya_fieldTbl=, M3_Bd56fi_Dir=, > M3_Bd56fi_SrcDir=, M3_AZx9O5_binaryFields=, M3_CebKnt_specialFields=, > M3_CT76zs_checkedTypes=) at ../AMD64_LINUX/Fields.m3:538 > #15 0x0000000000413948 in Main_M3 (M3_AcxOUs_mode=) at ../AMD64_LINUX/Main.m3:431 > #16 0x00000000004742ad in RTLinker__RunMainBody (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:408 > #17 0x0000000000473638 in RTLinker__AddUnitI (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:115 > #18 0x00000000004736cc in RTLinker__AddUnit (M3_DjPxE5_b=) at ../src/runtime/common/RTLinker.m3:124 > #19 0x0000000000405df8 in main (argc=8, argv=0x7fffffffe2b8, envp=0x7fffffffe300) at _m3main.c:22 > (gdb) > > I believe all the TextStats operations are killing the performance. Can we remove them? > > Here is SetChars: > > PROCEDURE SetChars (VAR a: ARRAY OF CHAR; t: T; start: CARDINAL) = > BEGIN > TextStats.NoteGround (TextStats.Op.SetChars); > TextStats.NoteGround (TextStats.Op.get_chars); > t.get_chars (a, start); > TextStats.NoteFinished (TextStats.Op.get_chars); > TextStats.NoteFinished (TextStats.Op.SetChars); > END SetChars; > > I'm not sure how best to do this. Removing them without killing functionality might not be > easy. Looks like a case for conditional compilation. Now the instrumentation calls are commented out with (*47 ... 74*), which could be reinstated by string search & replace "(*47" -> "(*47*)", etc., and removed again similarly. These are only needed for performance study of the algorithms, which likely won't be done often. > > Maybe generate the .m3s from some other source, with and without the TextStats, depending on > a setting in m3makefile? > > Second best would be to have a boolean flag in Text.m3 so the TextStats never get called. > > Mika > > From jay.krell at cornell.edu Fri Jan 10 09:22:25 2014 From: jay.krell at cornell.edu (Jay K) Date: Fri, 10 Jan 2014 08:22:25 +0000 Subject: [M3devel] performance problems in CM3 Text operations on AMD64_LINUX In-Reply-To: <52CF0E43.2030701@lcwb.coop> References: <20140104185511.9000A1A2080@async.async.caltech.edu>, <52CF0E43.2030701@lcwb.coop> Message-ID: Ok either, way, but here is another point. In some of our code we do: CONST DEBUG = FALSE; IF FALSE THEN xxEND; It will tend to be optimized.Furthermore, syscalls -- even if not optimized,IF FALSE is going to be way faster than a syscall. Also, gettimeofday shouldn't necessarily be a syscall. But/and there are/should-be faster ways of getting time for benchmarking. On Windows, GetTickCount and GetTickCount64 just read a value out of memory.Even GetSystemTimeAsFileTime is just a read of a 64bit value from memory.(The memory is read only, at the same address in all processes, and updated bythe kernel.)x86 has rdtsc, a 64bit hardware cycle counter readable in one instruction. We should consider researching analogs to rdtsc across mips/powerpc/sparc/arm andprovide something in m3core. I think.Add it to the backend interface likely, for inlining.or maybe it is worth a function call, then just use #ifdefed C. Maybe Posix provides something that tends to be like this? Mika, how did you find this?Control-c every so often and look at callstack?Or a lot of stepping? - Jay : Thu, 9 Jan 2014 15:01:55 -0600 > From: rodney_bates at lcwb.coop > To: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] performance problems in CM3 Text operations on AMD64_LINUX > > > > On 01/04/2014 12:55 PM, mika at async.caltech.edu wrote: > > I have this program that generates Modula-3 code from XML. > > > > It uses Wr.PutText a lot, on small strings. > > > > It runs 10 times faster under PM3 on FreeBSD/i386 on a 2 GHz old Pentium > > III type computer than under CM3 (at head) on Debian/amd64 on a 4 GHz > > Core i7 4960X. > > > > (176)rover:~/t/calarm/fix/gen/src>time ../FreeBSD4/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml > > 0.419u 0.089s 0:00.70 70.0% 471+27608k 0+546io 0pf+0w > > > > (29)truffles:~/t/calarm/fix/gen/src>time ../AMD64_LINUX/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml > > 1.952u 0.232s 0:05.40 40.3% 0+0k 0+11504io 0pf+0w > > > > I believe the issue is that Text.m3 is making a ton of system calls! > > > > Here's a typical traceback: > > > > (gdb) where > > #0 0x00002aaaaaacb770 in ?? () > > #1 0x00002aaaaaacba1b in gettimeofday () > > #2 0x00002aaaac468f8a in gettimeofday () at ../sysdeps/unix/sysv/linux/x86_64/gettimeofday.S:37 > > #3 0x000000000048c10c in TimePosix__Now () at ../src/time/POSIX/TimePosixC.c:50 > > #4 0x000000000048b4d2 in Time__Now () at ../src/time/POSIX/TimePosix.m3:14 > > #5 0x000000000049cbcf in TextStats__NoteFinished (M3_ESffkp_o=) at ../src/text/TextStats.m3:64 > > #6 0x00000000004921de in Text__SetChars (M3_CKMnXU_a=, M3_Bd56fi_t=, > > M3_Cwb5VA_start=) at ../src/text/Text.m3:512 > > #7 0x0000000000441a26 in UnsafeWr__FastPutText (M3_BxxOH1_wr=, M3_Bd56fi_t=) at ../src/rw/Wr.m3:100 > > #8 0x000000000044192f in Wr__PutText (M3_BxxOH1_wr=, M3_Bd56fi_t=) at ../src/rw/Wr.m3:89 > > #9 0x000000000041627f in M3Syntax__TextSetPutList (M3_CT76zs_s=, M3_BxxOH1_wr=, > > M3_AicXUJ_quoted=) at ../src/M3Syntax.m3:129 > > #10 0x00000000004169cf in M3Syntax__EnumToText (M3_Bqnwti_e=, M3_ACGgGY_env=) at ../src/M3Syntax.m3:173 > > #11 0x000000000041791a in M3Syntax__UnstrToText (M3_AIPC2K_u=, M3_ACGgGY_env=) at ../src/M3Syntax.m3:295 > > #12 0x0000000000414ad4 in M3File__DumpInterface (M3_CjySNq_t=, M3_Bd56fi_dirPath=, > > M3_AicXUJ_updateMakefile=) at ../src/M3File.m3:78 > > #13 0x000000000040f3f9 in Fields__Do__MakeFieldsFiles.1150 () at ../AMD64_LINUX/Fields.m3:526 > > #14 0x000000000040a713 in Fields__Do (M3_DsEykq_xml=, M3_Bd56fi_versionString=, > > M3_Bd56fi_beginStringV=, M3_D9m5ya_fieldTbl=, M3_Bd56fi_Dir=, > > M3_Bd56fi_SrcDir=, M3_AZx9O5_binaryFields=, M3_CebKnt_specialFields=, > > M3_CT76zs_checkedTypes=) at ../AMD64_LINUX/Fields.m3:538 > > #15 0x0000000000413948 in Main_M3 (M3_AcxOUs_mode=) at ../AMD64_LINUX/Main.m3:431 > > #16 0x00000000004742ad in RTLinker__RunMainBody (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:408 > > #17 0x0000000000473638 in RTLinker__AddUnitI (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:115 > > #18 0x00000000004736cc in RTLinker__AddUnit (M3_DjPxE5_b=) at ../src/runtime/common/RTLinker.m3:124 > > #19 0x0000000000405df8 in main (argc=8, argv=0x7fffffffe2b8, envp=0x7fffffffe300) at _m3main.c:22 > > (gdb) > > > > I believe all the TextStats operations are killing the performance. Can we remove them? > > > > Here is SetChars: > > > > PROCEDURE SetChars (VAR a: ARRAY OF CHAR; t: T; start: CARDINAL) = > > BEGIN > > TextStats.NoteGround (TextStats.Op.SetChars); > > TextStats.NoteGround (TextStats.Op.get_chars); > > t.get_chars (a, start); > > TextStats.NoteFinished (TextStats.Op.get_chars); > > TextStats.NoteFinished (TextStats.Op.SetChars); > > END SetChars; > > > > I'm not sure how best to do this. Removing them without killing functionality might not be > > easy. Looks like a case for conditional compilation. > > Now the instrumentation calls are commented out with (*47 ... 74*), which could be reinstated > by string search & replace "(*47" -> "(*47*)", etc., and removed again similarly. These are > only needed for performance study of the algorithms, which likely won't be done often. > > > > > Maybe generate the .m3s from some other source, with and without the TextStats, depending on > > a setting in m3makefile? > > > > Second best would be to have a boolean flag in Text.m3 so the TextStats never get called. > > > > Mika > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Fri Jan 10 16:27:10 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Fri, 10 Jan 2014 07:27:10 -0800 Subject: [M3devel] performance problems in CM3 Text operations on AMD64_LINUX In-Reply-To: References: <20140104185511.9000A1A2080@async.async.caltech.edu>, <52CF0E43.2030701@lcwb.coop> Message-ID: <20140110152711.036F21A208F@async.async.caltech.edu> >Mika=2C how did you find this?Control-c every so often and look at callstac= >k?Or a lot of stepping? "Poor man's profiling" -- since profiling is usually broken in Modula-3. It worked back with SRC M3 but that was a while ago :-) 1. run program in gdb 2. close eyes 3. count to three 4. hit ctrl-C 5. open eyes (repeat a few times) "close eyes" is important! From dragisha at m3w.org Sun Jan 12 09:49:23 2014 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 12 Jan 2014 09:49:23 +0100 Subject: [M3devel] Conversion to Git Message-ID: <4B139BF3-A08A-4FAB-A37B-C8FFCCDC1D7B@m3w.org> Hi, I am at this again? Last night I did another full conversion, and now I am verifying results. After verification I will do one more, final, conversion. For best results, I would like to have emails/names of following people/authors. Please reply directly to me with this data. alexb darko dbenavid dragisha eiserlohpp hosking hudson jkrell khaeusler kschleiser m3 mand micha mika mm neels pmckinna rcoleburn rforb rillig rodney sk stsp thielema uamoore wagner -- Dragi?a Duri? dragisha at m3w.org -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Sun Jan 12 18:53:17 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sun, 12 Jan 2014 09:53:17 -0800 Subject: [M3devel] CM3 crashing Message-ID: <20140112175317.9F0031A2080@async.async.caltech.edu> Hi m3devel, Anyone have an idea what's going on here? I have program that compiles with PM3 (FreeBSD4): ... new source -> compiling ../src/Main.m3 "../src/Main.m3", line 15: warning: exception is never raised: OSError.E "../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted "../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure new opaque info -> recompiling HParseStd.m3 ... With CM3 at head (AMD64_LINUX (userthreads)), the following happens: --- building in AMD64_LINUX --- echo _PARSER_SOURCES +='"'`pwd`/../src/H.t'"' > H_t.tmpl echo _PARSER_SOURCES +='"'`pwd`/../src/H.l'"' > H_l.tmpl echo _PARSER_SOURCES +='"'`pwd`/../src/H.y'"' > H_y.tmpl new source -> compiling Main.m3 "../src/Main.m3", line 104: warning: potentially unhandled exception: RTAllocator.OutOfMemory "../src/Main.m3", line 138: warning: potentially unhandled exception: RTAllocator.OutOfMemory "../src/Main.m3", line 15: warning: exception is never raised: OSError.E "../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted "../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure *** *** runtime error: *** Attempt to reference an illegal memory location. *** file "../src/thread/POSIX/ThreadPosix.m3", line 1073 *** Abort (gdb) run -x Starting program: /big/home/mika/cm3-boot2.userthreads/bin/cm3 -x [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". --- building in AMD64_LINUX --- echo _PARSER_SOURCES +='"'`pwd`/../src/H.t'"' > H_t.tmpl echo _PARSER_SOURCES +='"'`pwd`/../src/H.l'"' > H_l.tmpl echo _PARSER_SOURCES +='"'`pwd`/../src/H.y'"' > H_y.tmpl new source -> compiling Main.m3 "../src/Main.m3", line 104: warning: potentially unhandled exception: RTAllocator.OutOfMemory "../src/Main.m3", line 138: warning: potentially unhandled exception: RTAllocator.OutOfMemory "../src/Main.m3", line 15: warning: exception is never raised: OSError.E "../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted "../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure Program received signal SIGSEGV, Segmentation fault. 0x000000000072da7c in RTHooks__Concat (M3_Bd56fi_t=, M3_Bd56fi_u=) at ../src/text/TextCat.m3:36 36 u.get_info (RInfo); (gdb) where #0 0x000000000072da7c in RTHooks__Concat (M3_Bd56fi_t=, M3_Bd56fi_u=) at ../src/text/TextCat.m3:36 #1 0x00000000006904d1 in M3CG_Check__PutErr (M3_CquYO2_u=, M3_Bd56fi_a=, M3_Bd56fi_b=, M3_Bd56fi_c=) at ../src/M3CG_Check.m3:190 #2 0x0000000000692133 in M3CG_Check__CheckProc (M3_CquYO2_self=, M3_BgUkgw_p=) at ../src/M3CG_Check.m3:513 #3 0x00000000006969e7 in M3CG_Check__load_procedure (M3_CquYO2_self=, M3_BgUkgw_p=) at ../src/M3CG_Check.m3:1289 #4 0x000000000051c846 in CG__Load_procedure (M3_BgUkgw_p=) at ../src/misc/CG.m3:2547 #5 0x0000000000564b6d in Procedure__Load (M3_D4a7Im_t=) at ../src/values/Procedure.m3:358 #6 0x000000000054f81f in Value__Load (M3_EjfEr4_t=) at ../src/values/Value.m3:61 #7 0x00000000005b3451 in ProcExpr__Compile (M3_EeJ4NO_p=) at ../src/exprs/ProcExpr.m3:96 #8 0x0000000000596873 in Expr__Compile (M3_ES44mX_t=) at ../src/exprs/Expr.m3:145 #9 0x00000000005588db in Formal__GenProcedure (M3_DfGE7n_t=, M3_ES44mX_actual=, M3_ES44mX_proc=) at ../src/values/Formal.m3:550 #10 0x000000000055809c in Formal__EmitArg (M3_ES44mX_proc=, M3_EjfEr4_formal=, M3_ES44mX_actual=) at ../src/values/Formal.m3:459 #11 0x000000000054e82c in UserProc (M3_ChqBRb_ce=) at ../src/types/UserProc.m3:131 #12 0x00000000005a0de2 in CallExpr (M3_ChqBRb_p=) at ../src/exprs/CallExpr.m3:314 #13 0x0000000000596825 in Expr (M3_ES44mX_t=) at ../src/exprs/Expr.m3:138 #14 0x0000000000557e4d in Formal__PrepArg (M3_EjfEr4_formal=, M3_ES44mX_actual=) at ../src/values/Formal.m3:431 #15 0x000000000054e48e in UserProc (M3_ChqBRb_ce=) at ../src/types/UserProc.m3:91 #16 0x00000000005a0de2 in CallExpr (M3_ChqBRb_p=) at ../src/exprs/CallExpr.m3:314 #17 0x0000000000596825 in Expr (M3_ES44mX_t=) at ../src/exprs/Expr.m3:138 #18 0x00000000004f741e in Marker__EmitReturn (M3_ES44mX_expr=, M3_AicXUJ_fromFinally=) at ../src/misc/Marker.m3:540 #19 0x00000000005d9ed1 in ReturnStmt__Compile (M3_CnuIWR_p=) at ../src/stmts/ReturnStmt.m3:55 #20 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) at ../src/stmts/Stmt.m3:115 #21 0x00000000005dfdd5 in TypeCaseStmt__CompileCaseBody (M3_BVmtE4_p=, M3_CUZbO1_c=, M3_CFkyBH_ref=, M3_AcxOUs_case_label=, M3_AcxOUs_exit_label=, M3_EIM114_done=) at ../src/stmts/TypeCaseStmt.m3:345 #22 0x00000000005df778 in TypeCaseStmt__Compile (M3_BVmtE4_p=) at ../src/stmts/TypeCaseStmt.m3:284 #23 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) at ../src/stmts/Stmt.m3:115 #24 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=) at ../src/stmts/WithStmt.m3:150 #25 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) at ../src/stmts/Stmt.m3:115 #26 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=) at ../src/stmts/WithStmt.m3:150 #27 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) at ../src/stmts/Stmt.m3:115 #28 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=) at ../src/stmts/WithStmt.m3:150 #29 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) at ../src/stmts/Stmt.m3:115 #30 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=) at ../src/stmts/WithStmt.m3:150 #31 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) at ../src/stmts/Stmt.m3:115 #32 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=) at ../src/stmts/WithStmt.m3:150 #33 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) at ../src/stmts/Stmt.m3:115 #34 0x00000000005d7dd9 in IfStmt__Compile (M3_EFMY1O_p=) at ../src/stmts/IfStmt.m3:105 #35 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) at ../src/stmts/Stmt.m3:115 #36 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=) at ../src/stmts/WithStmt.m3:150 #37 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) at ../src/stmts/Stmt.m3:115 #38 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=) at ../src/stmts/WithStmt.m3:150 #39 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) at ../src/stmts/Stmt.m3:115 #40 0x00000000005dfcfa in TypeCaseStmt__CompileCaseBody (M3_BVmtE4_p=, M3_CUZbO1_c=, M3_CFkyBH_ref=, M3_AcxOUs_case_label=, M3_AcxOUs_exit_label=, M3_EIM114_done=) at ../src/stmts/TypeCaseStmt.m3:339 #41 0x00000000005df778 in TypeCaseStmt__Compile (M3_BVmtE4_p=) at ../src/stmts/TypeCaseStmt.m3:284 #42 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) at ../src/stmts/Stmt.m3:115 #43 0x00000000005ce5f3 in BlockStmt__Compile (M3_CwhzFF_p=) at ../src/stmts/BlockStmt.m3:103 #44 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) at ../src/stmts/Stmt.m3:115 #45 0x00000000005667a9 in Procedure__GenBody (M3_D4a7Im_p=) at ../src/values/Procedure.m3:577 #46 0x000000000056631d in Procedure__EmitBody (M3_CknzAQ_x=) at ../src/values/Procedure.m3:541 #47 0x0000000000509703 in ProcBody__EmitBody (M3_BswZye_t=) at ../src/misc/ProcBody.m3:170 #48 0x000000000050887f in ProcBody__EmitAll (M3_EN2A1V_proc_info=) at ../src/misc/ProcBody.m3:69 #49 0x00000000005603d4 in Module__CompileModule (M3_DZ1mTg_t=) at ../src/values/Module.m3:905 #50 0x000000000055fc79 in Module__Compile (M3_DZ1mTg_t=) at ../src/values/Module.m3:837 #51 0x00000000004ec678 in M3Front__DoCompile () at ../src/misc/M3Front.m3:218 #52 0x00000000004eb7e9 in M3Front__Compile (M3_Algh87_input=, M3_CT7GjM_env=, M3_BySCJy_options=) at ../src/misc/M3Front.m3:66 #53 0x0000000000411db9 in Builder__RunM3 (M3_Cwb8uG_s=, M3_AXLf8B_u=, M3_Bd56fi_object=) at ../src/Builder.m3:1705 #54 0x000000000040ff74 in Builder__PushOneM3 (M3_Cwb8uG_s=, M3_AXLf8B_u=) at ../src/Builder.m3:1359 #55 0x000000000040f5ca in Builder__CompileM3 (M3_Cwb8uG_s=, M3_AXLf8B_u=) at ../src/Builder.m3:1247 #56 0x000000000040de26 in Builder__CompileOne (M3_Cwb8uG_s=, M3_AXLf8B_u=) at ../src/Builder.m3:1077 #57 0x000000000040d7b7 in Builder__CompileEverything (M3_Cwb8uG_s=, M3_Cw4bpV_schedule=) at ../src/Builder.m3:1025 #58 0x0000000000408676 in Builder__CompileUnits (M3_Bd56fi_main=, M3_EEuw3X_units=, M3_C1FTrk_sys_libs=, M3_A2QN6Z_info_kind=, M3_An02H2_mach=) at ../src/Builder.m3:336 #59 0x0000000000406c0b in Builder__BuildPgm (M3_Bd56fi_prog=, M3_EEuw3X_units=, M3_C1FTrk_sys_libs=, M3_AicXUJ_shared=, M3_An02H2_m=) at ../src/Builder.m3:28 #60 0x00000000004256f3 in M3Build__BuildProgram (M3_ABp1Zk_t=, M3_DLS2Hj_nm=) at ../src/M3Build.m3:1515 #61 0x000000000042553b in M3Build__DoProgram (M3_An02H2_m=, M3_AcxOUs_n_args=) at ../src/M3Build.m3:1491 #62 0x00000000004d01ad in QMachine__DoCall (M3_An02H2_t=, M3_AcxOUs_n_args=, M3_AicXUJ_isFunc=, M3_AicXUJ_outer=) at ../src/QMachine.m3:546 #63 0x00000000004cee3f in QMachine__Eval (M3_An02H2_t=) at ../src/QMachine.m3:422 #64 0x00000000004d9152 in QMachine (M3_An02H2_t=, M3_Bd56fi_path=, M3_AicXUJ_from_code=) at ../src/QMachine.m3:1773 #65 0x00000000004d8f2f in QMachine (M3_An02H2_t=, M3_Bd56fi_path=) at ../src/QMachine.m3:1744 #66 0x0000000000420f0d in M3Build (M3_ABp1Zk_t=, M3_Bd56fi_file=) at ../src/M3Build.m3:679 #67 0x0000000000422dc9 in M3Build__DoIncludeDir (M3_An02H2_m=, M3_AcxOUs_n_args=) at ../src/M3Build.m3:1054 #68 0x00000000004d01ad in QMachine__DoCall (M3_An02H2_t=, M3_AcxOUs_n_args=, M3_AicXUJ_isFunc=, M3_AicXUJ_outer=) at ../src/QMachine.m3:546 #69 0x00000000004cee3f in QMachine__Eval (M3_An02H2_t=) at ../src/QMachine.m3:422 #70 0x00000000004cd167 in QMachine__Evaluate (M3_An02H2_t=, M3_CYwAos_s=) at ../src/QMachine.m3:165 #71 0x00000000004c3c18 in Quake__Run (M3_An02H2_m=, M3_Bd56fi_source_file=) at ../src/Quake.m3:19 #72 0x000000000041f10d in M3Build__Run (M3_ABp1Zk_t=, M3_Bd56fi_makefile=) at ../src/M3Build.m3:242 #73 0x00000000004372ed in Main__DoIt () at ../src/Main.m3:117 #74 0x0000000000437671 in Main_M3 (M3_AcxOUs_mode=) at ../src/Main.m3:231 #75 0x00000000007087ed in RTLinker__RunMainBody (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:408 #76 0x0000000000707b78 in RTLinker__AddUnitI (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:115 #77 0x0000000000707c0c in RTLinker__AddUnit (M3_DjPxE5_b=) at ../src/runtime/common/RTLinker.m3:124 #78 0x0000000000405b48 in main (argc=2, argv=0x7fffffffe378, envp=0x7fffffffe390) at _m3main.c:22 (gdb) Looks like something very basic... From mika at async.caltech.edu Sun Jan 12 18:58:30 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sun, 12 Jan 2014 09:58:30 -0800 Subject: [M3devel] CM3 crashing In-Reply-To: <20140112175317.9F0031A2080@async.async.caltech.edu> References: <20140112175317.9F0031A2080@async.async.caltech.edu> Message-ID: <20140112175830.C51881A2080@async.async.caltech.edu> Sigh... 188 PROCEDURE PutErr (u: U; a, b, c: TEXT := NIL) = 189 BEGIN 190 u.note_error ("********* M3CG_Check ERROR *********** " & a & b & c); 191 u.child.comment ("********* M3CG_Check ERROR *********** ", a, b, c); 192 INC (u.n_errors); 193 END PutErr; Ah.... ok.... "../src/Main.m3", line 129: ********* M3CG_Check ERROR *********** NIL procedure**NIL****NIL** "../src/Main.m3", line 131: ********* M3CG_Check ERROR *********** NIL procedure**NIL****NIL** "../src/Main.m3", line 133: ********* M3CG_Check ERROR *********** NIL procedure**NIL****NIL** " The code is: HIntExpr.Xor => RETURN NewConst(CBitwise(av, bv, Word.Xor), ab) | HIntExpr.Bor => RETURN NewConst(CBitwise(av, bv, Word.Or), ab) | HIntExpr.Band => RETURN NewConst(CBitwise(av, bv, Word.And), ab) I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ... mika writes: >Hi m3devel, > >Anyone have an idea what's going on here? > >I have program that compiles with PM3 (FreeBSD4): > >... >new source -> compiling ../src/Main.m3 >"../src/Main.m3", line 15: warning: exception is never raised: OSError.E >"../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted >"../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure > >new opaque info -> recompiling HParseStd.m3 >... > >With CM3 at head (AMD64_LINUX (userthreads)), the following happens: > >--- building in AMD64_LINUX --- > >echo _PARSER_SOURCES +='"'`pwd`/../src/H.t'"' > H_t.tmpl >echo _PARSER_SOURCES +='"'`pwd`/../src/H.l'"' > H_l.tmpl >echo _PARSER_SOURCES +='"'`pwd`/../src/H.y'"' > H_y.tmpl >new source -> compiling Main.m3 >"../src/Main.m3", line 104: warning: potentially unhandled exception: RTAlloca >tor.OutOfMemory >"../src/Main.m3", line 138: warning: potentially unhandled exception: RTAlloca >tor.OutOfMemory >"../src/Main.m3", line 15: warning: exception is never raised: OSError.E >"../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted >"../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure > > >*** >*** runtime error: >*** Attempt to reference an illegal memory location. >*** file "../src/thread/POSIX/ThreadPosix.m3", line 1073 >*** > >Abort > >(gdb) run -x >Starting program: /big/home/mika/cm3-boot2.userthreads/bin/cm3 -x >[Thread debugging using libthread_db enabled] >Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". >--- building in AMD64_LINUX --- > >echo _PARSER_SOURCES +='"'`pwd`/../src/H.t'"' > H_t.tmpl >echo _PARSER_SOURCES +='"'`pwd`/../src/H.l'"' > H_l.tmpl >echo _PARSER_SOURCES +='"'`pwd`/../src/H.y'"' > H_y.tmpl >new source -> compiling Main.m3 >"../src/Main.m3", line 104: warning: potentially unhandled exception: RTAlloca >tor.OutOfMemory >"../src/Main.m3", line 138: warning: potentially unhandled exception: RTAlloca >tor.OutOfMemory >"../src/Main.m3", line 15: warning: exception is never raised: OSError.E >"../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted >"../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure > >Program received signal SIGSEGV, Segmentation fault. >0x000000000072da7c in RTHooks__Concat (M3_Bd56fi_t=, M >3_Bd56fi_u=) at ../src/text/TextCat.m3:36 >36 u.get_info (RInfo); >(gdb) where >#0 0x000000000072da7c in RTHooks__Concat (M3_Bd56fi_t=>, M3_Bd56fi_u=) at ../src/text/TextCat.m3:36 >#1 0x00000000006904d1 in M3CG_Check__PutErr (M3_CquYO2_u=ble>, M3_Bd56fi_a=, M3_Bd56fi_b=e>, M3_Bd56fi_c=) at ../src/M3CG_Check.m3:190 >#2 0x0000000000692133 in M3CG_Check__CheckProc (M3_CquYO2_self= variable>, M3_BgUkgw_p=) at ../src/M3CG_Check.m3:513 >#3 0x00000000006969e7 in M3CG_Check__load_procedure (M3_CquYO2_self=ading variable>, M3_BgUkgw_p=) at ../src/M3CG_Check.m3 >:1289 >#4 0x000000000051c846 in CG__Load_procedure (M3_BgUkgw_p=ble>) at ../src/misc/CG.m3:2547 >#5 0x0000000000564b6d in Procedure__Load (M3_D4a7Im_t=>) at ../src/values/Procedure.m3:358 >#6 0x000000000054f81f in Value__Load (M3_EjfEr4_t=) a >t ../src/values/Value.m3:61 >#7 0x00000000005b3451 in ProcExpr__Compile (M3_EeJ4NO_p=le>) at ../src/exprs/ProcExpr.m3:96 >#8 0x0000000000596873 in Expr__Compile (M3_ES44mX_t=) > at ../src/exprs/Expr.m3:145 >#9 0x00000000005588db in Formal__GenProcedure (M3_DfGE7n_t=iable>, M3_ES44mX_actual=, M3_ES44mX_proc=ng variable>) at ../src/values/Formal.m3:550 >#10 0x000000000055809c in Formal__EmitArg (M3_ES44mX_proc=ble>, M3_EjfEr4_formal=, M3_ES44mX_actual=ng variable>) at ../src/values/Formal.m3:459 >#11 0x000000000054e82c in UserProc (M3_ChqBRb_ce=) at >../src/types/UserProc.m3:131 >#12 0x00000000005a0de2 in CallExpr (M3_ChqBRb_p=) at . >./src/exprs/CallExpr.m3:314 >#13 0x0000000000596825 in Expr (M3_ES44mX_t=) at ../sr >c/exprs/Expr.m3:138 >#14 0x0000000000557e4d in Formal__PrepArg (M3_EjfEr4_formal=iable>, M3_ES44mX_actual=) at ../src/values/Formal.m3: >431 >#15 0x000000000054e48e in UserProc (M3_ChqBRb_ce=) at >../src/types/UserProc.m3:91 >#16 0x00000000005a0de2 in CallExpr (M3_ChqBRb_p=) at . >./src/exprs/CallExpr.m3:314 >#17 0x0000000000596825 in Expr (M3_ES44mX_t=) at ../sr >c/exprs/Expr.m3:138 >#18 0x00000000004f741e in Marker__EmitReturn (M3_ES44mX_expr=riable>, M3_AicXUJ_fromFinally=) at ../src/misc/Marker >.m3:540 >#19 0x00000000005d9ed1 in ReturnStmt__Compile (M3_CnuIWR_p=able>) at ../src/stmts/ReturnStmt.m3:55 >#20 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) > at ../src/stmts/Stmt.m3:115 >#21 0x00000000005dfdd5 in TypeCaseStmt__CompileCaseBody (M3_BVmtE4_p=ading variable>, M3_CUZbO1_c=, M3_CFkyBH_ref=ading variable>, M3_AcxOUs_case_label=, > M3_AcxOUs_exit_label=, M3_EIM114_done=ng variable>) at ../src/stmts/TypeCaseStmt.m3:345 >#22 0x00000000005df778 in TypeCaseStmt__Compile (M3_BVmtE4_p=riable>) at ../src/stmts/TypeCaseStmt.m3:284 >#23 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) > at ../src/stmts/Stmt.m3:115 >#24 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=le>) at ../src/stmts/WithStmt.m3:150 >#25 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) > at ../src/stmts/Stmt.m3:115 >#26 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=le>) at ../src/stmts/WithStmt.m3:150 >#27 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) > at ../src/stmts/Stmt.m3:115 >#28 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=le>) at ../src/stmts/WithStmt.m3:150 >#29 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) > at ../src/stmts/Stmt.m3:115 >#30 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=le>) at ../src/stmts/WithStmt.m3:150 >#31 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) > at ../src/stmts/Stmt.m3:115 >#32 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=le>) at ../src/stmts/WithStmt.m3:150 >#33 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) > at ../src/stmts/Stmt.m3:115 >#34 0x00000000005d7dd9 in IfStmt__Compile (M3_EFMY1O_p=>) at ../src/stmts/IfStmt.m3:105 >#35 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) > at ../src/stmts/Stmt.m3:115 >#36 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=le>) at ../src/stmts/WithStmt.m3:150 >#37 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) > at ../src/stmts/Stmt.m3:115 >#38 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=le>) at ../src/stmts/WithStmt.m3:150 >#39 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) > at ../src/stmts/Stmt.m3:115 >#40 0x00000000005dfcfa in TypeCaseStmt__CompileCaseBody (M3_BVmtE4_p=ading variable>, M3_CUZbO1_c=, M3_CFkyBH_ref=ading variable>, M3_AcxOUs_case_label=, > M3_AcxOUs_exit_label=, M3_EIM114_done=ng variable>) at ../src/stmts/TypeCaseStmt.m3:339 >#41 0x00000000005df778 in TypeCaseStmt__Compile (M3_BVmtE4_p=riable>) at ../src/stmts/TypeCaseStmt.m3:284 >#42 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) > at ../src/stmts/Stmt.m3:115 >#43 0x00000000005ce5f3 in BlockStmt__Compile (M3_CwhzFF_p=ble>) at ../src/stmts/BlockStmt.m3:103 >#44 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) > at ../src/stmts/Stmt.m3:115 >#45 0x00000000005667a9 in Procedure__GenBody (M3_D4a7Im_p=ble>) at ../src/values/Procedure.m3:577 >#46 0x000000000056631d in Procedure__EmitBody (M3_CknzAQ_x=able>) at ../src/values/Procedure.m3:541 >#47 0x0000000000509703 in ProcBody__EmitBody (M3_BswZye_t=ble>) at ../src/misc/ProcBody.m3:170 >#48 0x000000000050887f in ProcBody__EmitAll (M3_EN2A1V_proc_info=g variable>) at ../src/misc/ProcBody.m3:69 >#49 0x00000000005603d4 in Module__CompileModule (M3_DZ1mTg_t=riable>) at ../src/values/Module.m3:905 >#50 0x000000000055fc79 in Module__Compile (M3_DZ1mTg_t=>) at ../src/values/Module.m3:837 >#51 0x00000000004ec678 in M3Front__DoCompile () at ../src/misc/M3Front.m3:218 >#52 0x00000000004eb7e9 in M3Front__Compile (M3_Algh87_input=iable>, M3_CT7GjM_env=, M3_BySCJy_options=ng variable>) at ../src/misc/M3Front.m3:66 >#53 0x0000000000411db9 in Builder__RunM3 (M3_Cwb8uG_s= >, M3_AXLf8B_u=, M3_Bd56fi_object=le>) at ../src/Builder.m3:1705 >#54 0x000000000040ff74 in Builder__PushOneM3 (M3_Cwb8uG_s=ble>, M3_AXLf8B_u=) at ../src/Builder.m3:1359 >#55 0x000000000040f5ca in Builder__CompileM3 (M3_Cwb8uG_s=ble>, M3_AXLf8B_u=) at ../src/Builder.m3:1247 >#56 0x000000000040de26 in Builder__CompileOne (M3_Cwb8uG_s=able>, M3_AXLf8B_u=) at ../src/Builder.m3:1077 >#57 0x000000000040d7b7 in Builder__CompileEverything (M3_Cwb8uG_s=ng variable>, M3_Cw4bpV_schedule=) at ../src/Builder.m >3:1025 >#58 0x0000000000408676 in Builder__CompileUnits (M3_Bd56fi_main= variable>, M3_EEuw3X_units=, M3_C1FTrk_sys_libs=r reading variable>, M3_A2QN6Z_info_kind=, > M3_An02H2_mach=) at ../src/Builder.m3:336 >#59 0x0000000000406c0b in Builder__BuildPgm (M3_Bd56fi_prog=iable>, M3_EEuw3X_units=, M3_C1FTrk_sys_libs=ading variable>, M3_AicXUJ_shared=, > M3_An02H2_m=) at ../src/Builder.m3:28 >#60 0x00000000004256f3 in M3Build__BuildProgram (M3_ABp1Zk_t=riable>, M3_DLS2Hj_nm=) at ../src/M3Build.m3:1515 >#61 0x000000000042553b in M3Build__DoProgram (M3_An02H2_m=ble>, M3_AcxOUs_n_args=) at ../src/M3Build.m3:1491 >#62 0x00000000004d01ad in QMachine__DoCall (M3_An02H2_t=e>, M3_AcxOUs_n_args=, M3_AicXUJ_isFunc= variable>, M3_AicXUJ_outer=) > at ../src/QMachine.m3:546 >#63 0x00000000004cee3f in QMachine__Eval (M3_An02H2_t= >) at ../src/QMachine.m3:422 >#64 0x00000000004d9152 in QMachine (M3_An02H2_t=, M3_B >d56fi_path=, M3_AicXUJ_from_code=le>) at ../src/QMachine.m3:1773 >#65 0x00000000004d8f2f in QMachine (M3_An02H2_t=, M3_B >d56fi_path=) at ../src/QMachine.m3:1744 >#66 0x0000000000420f0d in M3Build (M3_ABp1Zk_t=, M3_Bd >56fi_file=) at ../src/M3Build.m3:679 >#67 0x0000000000422dc9 in M3Build__DoIncludeDir (M3_An02H2_m=riable>, M3_AcxOUs_n_args=) at ../src/M3Build.m3:1054 >#68 0x00000000004d01ad in QMachine__DoCall (M3_An02H2_t=e>, M3_AcxOUs_n_args=, M3_AicXUJ_isFunc= variable>, M3_AicXUJ_outer=) > at ../src/QMachine.m3:546 >#69 0x00000000004cee3f in QMachine__Eval (M3_An02H2_t= >) at ../src/QMachine.m3:422 >#70 0x00000000004cd167 in QMachine__Evaluate (M3_An02H2_t=ble>, M3_CYwAos_s=) at ../src/QMachine.m3:165 >#71 0x00000000004c3c18 in Quake__Run (M3_An02H2_m=, M3 >_Bd56fi_source_file=) at ../src/Quake.m3:19 >#72 0x000000000041f10d in M3Build__Run (M3_ABp1Zk_t=, >M3_Bd56fi_makefile=) at ../src/M3Build.m3:242 >#73 0x00000000004372ed in Main__DoIt () at ../src/Main.m3:117 >#74 0x0000000000437671 in Main_M3 (M3_AcxOUs_mode=) at > ../src/Main.m3:231 >#75 0x00000000007087ed in RTLinker__RunMainBody (M3_DjPxE3_m=riable>) at ../src/runtime/common/RTLinker.m3:408 >#76 0x0000000000707b78 in RTLinker__AddUnitI (M3_DjPxE3_m=ble>) at ../src/runtime/common/RTLinker.m3:115 >#77 0x0000000000707c0c in RTLinker__AddUnit (M3_DjPxE5_b=le>) at ../src/runtime/common/RTLinker.m3:124 >#78 0x0000000000405b48 in main (argc=2, argv=0x7fffffffe378, envp=0x7fffffffe3 >90) at _m3main.c:22 >(gdb) > >Looks like something very basic... > > From hosking at cs.purdue.edu Sun Jan 12 21:30:34 2014 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 12 Jan 2014 15:30:34 -0500 Subject: [M3devel] CM3 crashing In-Reply-To: <20140112175830.C51881A2080@async.async.caltech.edu> References: <20140112175317.9F0031A2080@async.async.caltech.edu> <20140112175830.C51881A2080@async.async.caltech.edu> Message-ID: Are you saying passing these used to work in PM3? Sounds like a front-end bug. I?m curious what changed to break it. On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: > > The code is: > > HIntExpr.Xor => RETURN NewConst(CBitwise(av, bv, Word.Xor), ab) > | > HIntExpr.Bor => RETURN NewConst(CBitwise(av, bv, Word.Or), ab) > | > HIntExpr.Band => RETURN NewConst(CBitwise(av, bv, Word.And), ab) > > I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ... Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Mobile +1 765 427 5484 From dragisha at m3w.org Sun Jan 12 22:45:32 2014 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 12 Jan 2014 22:45:32 +0100 Subject: [M3devel] Conversion to Git In-Reply-To: <4B139BF3-A08A-4FAB-A37B-C8FFCCDC1D7B@m3w.org> References: <4B139BF3-A08A-4FAB-A37B-C8FFCCDC1D7B@m3w.org> Message-ID: Thanks to everybody who send me infomation, esp. to Peter Eiserloh who gave me information + idea how to fill rest. Missing are emails from some Elego people and I expect I will have that soon. I will keep you all posted on conversion status. Current version is on Github (dragisha/cm3), and I am checking how it matches with local CVS checkouts. On 12 Jan 2014, at 09:49, Dragi?a Duri? wrote: > Hi, > > I am at this again? Last night I did another full conversion, and now I am verifying results. After verification I will do one more, final, conversion. > > For best results, I would like to have emails/names of following people/authors. Please reply directly to me with this data. > > alexb > darko > dbenavid > dragisha > eiserlohpp > hosking > hudson > jkrell > khaeusler > kschleiser > m3 > mand > micha > mika > mm > neels > pmckinna > rcoleburn > rforb > rillig > rodney > sk > stsp > thielema > uamoore > wagner > -- > Dragi?a Duri? > dragisha at m3w.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Mon Jan 13 01:25:39 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sun, 12 Jan 2014 16:25:39 -0800 Subject: [M3devel] CM3 crashing In-Reply-To: References: <20140112175317.9F0031A2080@async.async.caltech.edu> <20140112175830.C51881A2080@async.async.caltech.edu> Message-ID: <20140113002539.EC4EE1A2080@async.async.caltech.edu> Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3). I figured there aren't actually implementations of Word.Xor, Word.Or, and Word.And as procedures but they get inlined somehow? BTW, any idea about what's wrong with pthreads? Do you think the issue is with the garbage collector or with the threads, off the top of your head? Tony Hosking writes: >Are you saying passing these used to work in PM3? >Sounds like a front-end bug. I=92m curious what changed to break it. > >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: > >>=20 >> The code is: >>=20 >> HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Xor), ab) >> | >> HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Or), ab) >> | >> HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, = >Word.And), ab) >>=20 >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20 > > > >Antony Hosking | Associate Professor | Computer Science | Purdue = >University >305 N. University Street | West Lafayette | IN 47907 | USA >Mobile +1 765 427 5484 > > > > From hosking at cs.purdue.edu Mon Jan 13 16:46:36 2014 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 13 Jan 2014 10:46:36 -0500 Subject: [M3devel] CM3 crashing In-Reply-To: <20140113002539.EC4EE1A2080@async.async.caltech.edu> References: <20140112175317.9F0031A2080@async.async.caltech.edu> <20140112175830.C51881A2080@async.async.caltech.edu> <20140113002539.EC4EE1A2080@async.async.caltech.edu> Message-ID: <09749A02-B9B9-4C74-8CB4-73379E91385B@cs.purdue.edu> On Jan 12, 2014, at 7:25 PM, mika at async.caltech.edu wrote: > Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3). Interesting. Needs exploration. This should be filed as a bug. The workaround is to wrap the builtin calls (Word.Xor, etc.) in a local procedure so that you have an addressable procedure. > I figured there aren't actually implementations of Word.Xor, Word.Or, > and Word.And as procedures but they get inlined somehow? Yes, they are inlined, but there *is* an implementation in Word.m3 (generated from GenWord.mg). > BTW, any idea about what's wrong with pthreads? Do you think the issue > is with the garbage collector or with the threads, off the top of your > head? It must be the threads implementation, since the same collector is used with both pthreads and user threads, and I believe that user threads don?t suffer from this problem. My intuition is that it may be a problem with getting stack roots on particular platforms, probably because the stack bounds are not being computed correctly for a given architecture, or perhaps because reference alignment is not being observed correctly. For example, is it possible on x86_64 to have references stored in the stack aligned on 32-bit instead of 64-bit boundaries? I notice that RTThread.i3 says the pointer alignment is BYTESIZE(ADDRESS) on all platforms. Once upon a time the alignment was platform-dependent, so I wonder if we lost something somewhere with all the backend and porting work. It may be that the backend needs to be instructed to align all references appropriately in the stacks. One way to test all this might be to see if the thread stress tester causes things to break on older 32-bit platforms, where I expect all pointers will be aligned properly. Also, I?ve not inspected the code in ThreadPThreadC.c recently (it?s changed since I last looked), but I wonder if the stack bounds are being computed correctly so that the collector can find all the stack roots. I?d really like to get to the bottom of this problem. > Tony Hosking writes: >> Are you saying passing these used to work in PM3? >> Sounds like a front-end bug. I=92m curious what changed to break it. >> >> On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: >> >>> =20 >>> The code is: >>> =20 >>> HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, = >> Word.Xor), ab) >>> | >>> HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, = >> Word.Or), ab) >>> | >>> HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, = >> Word.And), ab) >>> =20 >>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20 >> >> >> >> Antony Hosking | Associate Professor | Computer Science | Purdue = >> University >> 305 N. University Street | West Lafayette | IN 47907 | USA >> Mobile +1 765 427 5484 >> >> >> >> From hosking at cs.purdue.edu Mon Jan 13 17:03:38 2014 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 13 Jan 2014 11:03:38 -0500 Subject: [M3devel] CM3 crashing In-Reply-To: References: <20140112175317.9F0031A2080@async.async.caltech.edu> <20140112175830.C51881A2080@async.async.caltech.edu> <20140113002539.EC4EE1A2080@async.async.caltech.edu> Message-ID: Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?). In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem. As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC. Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning. Can someone confirm? If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed. On Jan 12, 2014, at 10:54 PM, Peter McKinna wrote: > Hey, > > I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. > My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped the second call to p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one. > It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. > > Regards Peter > > > > On Mon, Jan 13, 2014 at 11:25 AM, wrote: > Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3). > > I figured there aren't actually implementations of Word.Xor, Word.Or, > and Word.And as procedures but they get inlined somehow? > > BTW, any idea about what's wrong with pthreads? Do you think the issue > is with the garbage collector or with the threads, off the top of your > head? > > Tony Hosking writes: > >Are you saying passing these used to work in PM3? > >Sounds like a front-end bug. I=92m curious what changed to break it. > > > >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: > > > >>=20 > >> The code is: > >>=20 > >> HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, = > >Word.Xor), ab) > >> | > >> HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, = > >Word.Or), ab) > >> | > >> HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, = > >Word.And), ab) > >>=20 > >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20 > > > > > > > >Antony Hosking | Associate Professor | Computer Science | Purdue = > >University > >305 N. University Street | West Lafayette | IN 47907 | USA > >Mobile +1 765 427 5484 > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcolebur at SCIRES.COM Mon Jan 13 20:16:41 2014 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Mon, 13 Jan 2014 19:16:41 +0000 Subject: [M3devel] CM3 crashing Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D152@ATLEX04-SRV.SCIRES.LOCAL> Tony et al: The thread test program fails for me on 64-bit Windows 7. I've listed output from a couple of runs below: C:\cm3\Sandbox>cm3 --version Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2014-01-07 06:55:14 configuration: C:\cm3\bin\cm3.cfg host: NT386 target: NT386 FIRST RUN: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\runtime\common\RTCollector.m3", line 418 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** ...this last assertion repeats until you press CTRL-C to abort the program... SECOND RUN: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) .... *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x776f22d2 *** Stack trace: FP PC Procedure --------- --------- ------------------------------- 0xa6f9a8 0xc6afeb SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3 0xa6f9d0 0x776f22d2 0xa6f9e8 0xc4a39b LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3 0xa6fa10 0xc299f8 GetChar + 0x28 in ..\src\rw\Rd.m3 0xa6fb48 0xc213a8 RApply + 0x1b8 in ..\src\Main.m3 0xa6fb84 0xc4b71b ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3 0xa6fb90 0x76db336a 0xa6fbd0 0x7770bf32 ......... ......... ... more frames ... Maybe this info will be useful in tracking down the problem. --Randy Randy C. Coleburn, CISSP-ISSEP, GCED Corporate Information Security Officer (CISO) Scientific Research Corporation 2300 Windy Ridge Parkway, Suite 400 South, Atlanta, Georgia 30339 voice: (770) 989-9464, email: RColeburn at SciRes.com, fax: (770) 989-9497 [logo_SRC] "Technology Driven-Customer Focused" Quality Policy: "SRC is committed to delivering continually improving research & engineering excellence that meets or exceeds customer requirements." CONFIDENTIALITY NOTICE: This email constitutes an electronic communication within the meaning of the Electronic Communications Privacy Act, 18 U.S.C. 2510, and its disclosure is strictly limited to the named recipient(s) intended by the sender of this message. This email, and any attachments, may contain confidential and/or proprietary information of Scientific Research Corporation. If you are not a named recipient, any copying, using, disclosing or distributing to others the information in this email and attachments is STRICTLY PROHIBITED. If you have received this email in error, please notify the sender immediately and permanently delete the email, any attachments, and all copies thereof from any drives or storage media and destroy any printouts or hard copies of the email and attachments. EXPORT COMPLIANCE NOTICE: This email and any attachments may contain technical data subject to U.S export restrictions under the International Traffic in Arms Regulations (ITAR) or the Export Administration Regulations (EAR). Export or transfer of this technical data and/or related information to any foreign person(s) or entity(ies), either within the U.S. or outside of the U.S., may require advance export authorization by the appropriate U.S. Government agency prior to export or transfer. In addition, technical data may not be exported or transferred to certain countries or specified designated nationals identified by U.S. embargo controls without prior export authorization. By accepting this email and any attachments, all recipients confirm that they understand and will comply with all applicable ITAR, EAR and embargo compliance requirements. From: Tony Hosking [mailto:hosking at cs.purdue.edu] Sent: Monday, January 13, 2014 11:04 AM To: Peter McKinna Cc: m3devel Subject: EXT:Re: [M3devel] CM3 crashing Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?). In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem. As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC. Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning. Can someone confirm? If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed. On Jan 12, 2014, at 10:54 PM, Peter McKinna > wrote: Hey, I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped the second call to p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one. It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. Regards Peter On Mon, Jan 13, 2014 at 11:25 AM, > wrote: Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3). I figured there aren't actually implementations of Word.Xor, Word.Or, and Word.And as procedures but they get inlined somehow? BTW, any idea about what's wrong with pthreads? Do you think the issue is with the garbage collector or with the threads, off the top of your head? Tony Hosking writes: >Are you saying passing these used to work in PM3? >Sounds like a front-end bug. I=92m curious what changed to break it. > >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: > >>=20 >> The code is: >>=20 >> HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Xor), ab) >> | >> HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Or), ab) >> | >> HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, = >Word.And), ab) >>=20 >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20 > > > >Antony Hosking | Associate Professor | Computer Science | Purdue = >University >305 N. University Street | West Lafayette | IN 47907 | USA >Mobile +1 765 427 5484 > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.jpg Type: image/jpeg Size: 11339 bytes Desc: image002.jpg URL: From rcolebur at SCIRES.COM Mon Jan 13 20:55:44 2014 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Mon, 13 Jan 2014 19:55:44 +0000 Subject: [M3devel] CM3 crashing In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D152@ATLEX04-SRV.SCIRES.LOCAL> References: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D152@ATLEX04-SRV.SCIRES.LOCAL> Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D21A@ATLEX04-SRV.SCIRES.LOCAL> I've also tested on 32-bit Windows XP and the thread test program crashes there also. C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386 C:\cm3\Sandbox\cm3\m3-libs>cm3 --version Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2013-12-17 17:28:52 configuration: C:\cm3\bin\cm3.cfg host: NT386 target: NT386 Here is the output: C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Coleburn, Randy Sent: Monday, January 13, 2014 2:17 PM To: Tony Hosking; Peter McKinna Cc: m3devel Subject: Re: [M3devel] CM3 crashing Tony et al: The thread test program fails for me on 64-bit Windows 7. I've listed output from a couple of runs below: C:\cm3\Sandbox>cm3 --version Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2014-01-07 06:55:14 configuration: C:\cm3\bin\cm3.cfg host: NT386 target: NT386 FIRST RUN: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\runtime\common\RTCollector.m3", line 418 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** ...this last assertion repeats until you press CTRL-C to abort the program... SECOND RUN: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) .... *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x776f22d2 *** Stack trace: FP PC Procedure --------- --------- ------------------------------- 0xa6f9a8 0xc6afeb SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3 0xa6f9d0 0x776f22d2 0xa6f9e8 0xc4a39b LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3 0xa6fa10 0xc299f8 GetChar + 0x28 in ..\src\rw\Rd.m3 0xa6fb48 0xc213a8 RApply + 0x1b8 in ..\src\Main.m3 0xa6fb84 0xc4b71b ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3 0xa6fb90 0x76db336a 0xa6fbd0 0x7770bf32 ......... ......... ... more frames ... Maybe this info will be useful in tracking down the problem. --Randy From: Tony Hosking [mailto:hosking at cs.purdue.edu] Sent: Monday, January 13, 2014 11:04 AM To: Peter McKinna Cc: m3devel Subject: EXT:Re: [M3devel] CM3 crashing Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?). In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem. As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC. Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning. Can someone confirm? If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed. On Jan 12, 2014, at 10:54 PM, Peter McKinna > wrote: Hey, I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped the second call to p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one. It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. Regards Peter On Mon, Jan 13, 2014 at 11:25 AM, > wrote: Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3). I figured there aren't actually implementations of Word.Xor, Word.Or, and Word.And as procedures but they get inlined somehow? BTW, any idea about what's wrong with pthreads? Do you think the issue is with the garbage collector or with the threads, off the top of your head? Tony Hosking writes: >Are you saying passing these used to work in PM3? >Sounds like a front-end bug. I=92m curious what changed to break it. > >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: > >>=20 >> The code is: >>=20 >> HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Xor), ab) >> | >> HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Or), ab) >> | >> HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, = >Word.And), ab) >>=20 >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20 > > > >Antony Hosking | Associate Professor | Computer Science | Purdue = >University >305 N. University Street | West Lafayette | IN 47907 | USA >Mobile +1 765 427 5484 > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Mon Jan 13 21:02:57 2014 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 13 Jan 2014 15:02:57 -0500 Subject: [M3devel] CM3 crashing In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D21A@ATLEX04-SRV.SCIRES.LOCAL> References: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D152@ATLEX04-SRV.SCIRES.LOCAL> <0BB8FA59C2932741A3A2941A8B9D8BFF9251D21A@ATLEX04-SRV.SCIRES.LOCAL> Message-ID: <227E8C61-B070-49FF-8AD5-E23420E9C1B0@cs.purdue.edu> Is this from the head of source? Also, please run with @M3paranoidgc flag. On Jan 13, 2014, at 2:55 PM, Coleburn, Randy wrote: > I?ve also tested on 32-bit Windows XP and the thread test program crashes there also. > > C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386 > > C:\cm3\Sandbox\cm3\m3-libs>cm3 --version > Critical Mass Modula-3 version d5.9.0 > last updated: 2010-07-21 > compiled: 2013-12-17 17:28:52 > configuration: C:\cm3\bin\cm3.cfg > host: NT386 > target: NT386 > > Here is the output: > > C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help > Writing file...done > Creating read threads...done > Creating fork threads...done > Creating alloc threads...done > Creating lock threads...done > running...printing oldest/median age/newest > > *** > *** runtime error: > *** Attempt to reference an illegal memory location. > *** pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3 > *** > > *** > *** runtime error: > *** <*ASSERT*> failed. > *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 > *** > > *** > *** runtime error: > *** <*ASSERT*> failed. > *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 > *** > > This last assert repeats indefinitely until you press CTRL-C. > > --Randy > > > From: Coleburn, Randy > Sent: Monday, January 13, 2014 2:17 PM > To: Tony Hosking; Peter McKinna > Cc: m3devel > Subject: Re: [M3devel] CM3 crashing > > Tony et al: > > The thread test program fails for me on 64-bit Windows 7. > > I?ve listed output from a couple of runs below: > > C:\cm3\Sandbox>cm3 --version > Critical Mass Modula-3 version d5.9.0 > last updated: 2010-07-21 > compiled: 2014-01-07 06:55:14 > configuration: C:\cm3\bin\cm3.cfg > host: NT386 > target: NT386 > > FIRST RUN: > > C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe > Writing file...done > Creating read threads...done > Creating fork threads...done > Creating alloc threads...done > Creating lock threads...done > running...printing oldest/median age/newest > > *** > *** runtime error: > *** Attempt to reference an illegal memory location. > > *** > *** runtime error: > *** Attempt to reference an illegal memory location. > > *** > *** runtime error: > *** Attempt to reference an illegal memory location. > > *** > *** runtime error: > *** Attempt to reference an illegal memory location. > > *** > *** runtime error: > *** Attempt to reference an illegal memory location. > > *** > *** runtime error: > *** <*ASSERT*> failed. > *** file "..\src\runtime\common\RTCollector.m3", line 418 > *** > > *** > *** runtime error: > *** <*ASSERT*> failed. > *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 > *** > > *** > *** runtime error: > *** <*ASSERT*> failed. > *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 > *** > > ?this last assertion repeats until you press CTRL-C to abort the program? > > SECOND RUN: > > C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help > Writing file...done > Creating read threads...done > Creating fork threads...done > Creating alloc threads...done > Creating lock threads...done > running...printing oldest/median age/newest > ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) > .... > > *** > *** runtime error: > *** Attempt to reference an illegal memory location. > *** pc = 0x776f22d2 > *** > > Stack trace: > FP PC Procedure > --------- --------- ------------------------------- > 0xa6f9a8 0xc6afeb SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3 > 0xa6f9d0 0x776f22d2 > 0xa6f9e8 0xc4a39b LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3 > 0xa6fa10 0xc299f8 GetChar + 0x28 in ..\src\rw\Rd.m3 > 0xa6fb48 0xc213a8 RApply + 0x1b8 in ..\src\Main.m3 > 0xa6fb84 0xc4b71b ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3 > 0xa6fb90 0x76db336a > 0xa6fbd0 0x7770bf32 > ......... ......... ... more frames ... > > Maybe this info will be useful in tracking down the problem. > > --Randy > > > From: Tony Hosking [mailto:hosking at cs.purdue.edu] > Sent: Monday, January 13, 2014 11:04 AM > To: Peter McKinna > Cc: m3devel > Subject: EXT:Re: [M3devel] CM3 crashing > > Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?). > In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem. > > As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC. Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning. Can someone confirm? > > If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed. > > On Jan 12, 2014, at 10:54 PM, Peter McKinna wrote: > > > Hey, > > I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. > My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped the second call to p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one. > It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. > > Regards Peter > > > > On Mon, Jan 13, 2014 at 11:25 AM, wrote: > Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3). > > I figured there aren't actually implementations of Word.Xor, Word.Or, > and Word.And as procedures but they get inlined somehow? > > BTW, any idea about what's wrong with pthreads? Do you think the issue > is with the garbage collector or with the threads, off the top of your > head? > > Tony Hosking writes: > >Are you saying passing these used to work in PM3? > >Sounds like a front-end bug. I=92m curious what changed to break it. > > > >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: > > > >>=20 > >> The code is: > >>=20 > >> HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, = > >Word.Xor), ab) > >> | > >> HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, = > >Word.Or), ab) > >> | > >> HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, = > >Word.And), ab) > >>=20 > >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20 > > > > > > > >Antony Hosking | Associate Professor | Computer Science | Purdue = > >University > >305 N. University Street | West Lafayette | IN 47907 | USA > >Mobile +1 765 427 5484 > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcolebur at SCIRES.COM Mon Jan 13 21:13:55 2014 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Mon, 13 Jan 2014 20:13:55 +0000 Subject: [M3devel] CM3 crashing Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D28A@ATLEX04-SRV.SCIRES.LOCAL> Tony: Yes, it is from the HEAD as of 17 December 2013. Using the @M3paranoidgc flag, On 32-bit Windows XP I get: C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest . *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x431755 = RefSanityCheck + 0x2c in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. On 64-bit Windows 7 I get: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest . *** *** runtime error: *** An array subscript was out of range. *** file "..\src\rw\FileRd.m3", line 83 *** *** *** runtime error: *** An enumeration or subrange value was out of range. *** file "..\src\runtime\common\RTType.m3", line 71 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Tony Hosking [mailto:hosking at cs.purdue.edu] Sent: Monday, January 13, 2014 3:03 PM To: Coleburn, Randy Cc: Peter McKinna; m3devel Subject: EXT:Re: [M3devel] CM3 crashing Is this from the head of source? Also, please run with @M3paranoidgc flag. On Jan 13, 2014, at 2:55 PM, Coleburn, Randy > wrote: I've also tested on 32-bit Windows XP and the thread test program crashes there also. C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386 C:\cm3\Sandbox\cm3\m3-libs>cm3 --version Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2013-12-17 17:28:52 configuration: C:\cm3\bin\cm3.cfg host: NT386 target: NT386 Here is the output: C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Coleburn, Randy Sent: Monday, January 13, 2014 2:17 PM To: Tony Hosking; Peter McKinna Cc: m3devel Subject: Re: [M3devel] CM3 crashing Tony et al: The thread test program fails for me on 64-bit Windows 7. I've listed output from a couple of runs below: C:\cm3\Sandbox>cm3 --version Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2014-01-07 06:55:14 configuration: C:\cm3\bin\cm3.cfg host: NT386 target: NT386 FIRST RUN: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\runtime\common\RTCollector.m3", line 418 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** ...this last assertion repeats until you press CTRL-C to abort the program... SECOND RUN: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) .... *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x776f22d2 *** Stack trace: FP PC Procedure --------- --------- ------------------------------- 0xa6f9a8 0xc6afeb SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3 0xa6f9d0 0x776f22d2 0xa6f9e8 0xc4a39b LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3 0xa6fa10 0xc299f8 GetChar + 0x28 in ..\src\rw\Rd.m3 0xa6fb48 0xc213a8 RApply + 0x1b8 in ..\src\Main.m3 0xa6fb84 0xc4b71b ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3 0xa6fb90 0x76db336a 0xa6fbd0 0x7770bf32 ......... ......... ... more frames ... Maybe this info will be useful in tracking down the problem. --Randy From: Tony Hosking [mailto:hosking at cs.purdue.edu] Sent: Monday, January 13, 2014 11:04 AM To: Peter McKinna Cc: m3devel Subject: EXT:Re: [M3devel] CM3 crashing Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?). In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem. As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC. Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning. Can someone confirm? If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed. On Jan 12, 2014, at 10:54 PM, Peter McKinna > wrote: Hey, I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped the second call to p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one. It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. Regards Peter On Mon, Jan 13, 2014 at 11:25 AM, > wrote: Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3). I figured there aren't actually implementations of Word.Xor, Word.Or, and Word.And as procedures but they get inlined somehow? BTW, any idea about what's wrong with pthreads? Do you think the issue is with the garbage collector or with the threads, off the top of your head? Tony Hosking writes: >Are you saying passing these used to work in PM3? >Sounds like a front-end bug. I=92m curious what changed to break it. > >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: > >>=20 >> The code is: >>=20 >> HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Xor), ab) >> | >> HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Or), ab) >> | >> HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, = >Word.And), ab) >>=20 >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20 > > > >Antony Hosking | Associate Professor | Computer Science | Purdue = >University >305 N. University Street | West Lafayette | IN 47907 | USA >Mobile +1 765 427 5484 > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcolebur at SCIRES.COM Mon Jan 13 22:02:52 2014 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Mon, 13 Jan 2014 21:02:52 +0000 Subject: [M3devel] CM3 crashing In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D28A@ATLEX04-SRV.SCIRES.LOCAL> References: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D28A@ATLEX04-SRV.SCIRES.LOCAL> Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D3DF@ATLEX04-SRV.SCIRES.LOCAL> Tony: Here are a couple more runs on 64-bit Windows 7 that yield different results, albeit using the "-verbose" option. These may give some more clues. I note that in both cases it appears a read thread and a fork thread get the same ID (0) which looks very suspicious. By my read of the code (WITH id = i * nPer + j), I think the 2nd fork thread should be ID=10, not ID=0. Oops, now I see the problem: there is a bug in the PutInt procedure. The test (IF c > 10) should read instead (IF c >= 10). I'll commit a fix for that momentarily. But this fix just solves the output problem; it doesn't affect the fact that the program is misbehaving and crashing. RUN #1: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc -verbose Writing file...done Creating read threads... read=0 read=1 read=2 done Creating fork threads... fork=9 fork=0 fork=11 done Creating alloc threads... alloc=15 alloc=16 alloc=17 done Creating lock threads... lock=21 lock=22 lock=23 done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0xc286e0 = Init + 0x79 in ..\src\rw\FileRd.m3 *** Stack trace: FP PC Procedure --------- --------- ------------------------------- 0x7cfd60 0xc6afeb SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3 0x7cfda8 0xc286e0 Init + 0x79 in ..\src\rw\FileRd.m3 0x7cfdd4 0xc2865d Open + 0x4d in ..\src\rw\FileRd.m3 0x7cff0c 0xc21350 RApply + 0x160 in ..\src\Main.m3 0x7cff48 0xc4b71b ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3 0x7cff54 0x76db336a 0x7cff94 0x7770bf32 ......... ......... ... more frames ... RUN #2: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe -verbose Writing file...done Creating read threads... read=0 read=1 read=2 done Creating fork threads... fork=9 fork=0 fork=11 done Creating alloc threads... alloc=15 alloc=16 alloc=17 done Creating lock threads... lock=21 lock=22 lock=23 done running...printing oldest/median age/newest ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 534 loops. read Thread 1 completed 606 loops. read Thread 2 completed 398 loops. fork Thread 9 completed 8 loops. fork Thread 10 completed 8 loops. fork Thread 11 completed 8 loops. alloc Thread 15 completed 18296 loops. alloc Thread 16 completed 44871 loops. alloc Thread 17 completed 79125 loops. lock Thread 21 completed 3845080 loops. lock Thread 22 completed 2537613 loops. lock Thread 23 completed 4506151 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 790 loops. read Thread 1 completed 786 loops. read Thread 2 completed 675 loops. fork Thread 9 completed 9 loops. fork Thread 10 completed 9 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 22091 loops. alloc Thread 16 completed 47532 loops. alloc Thread 17 completed 68901 loops. lock Thread 21 completed 4560731 loops. lock Thread 22 completed 3440795 loops. lock Thread 23 completed 6433538 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 725 loops. read Thread 1 completed 705 loops. read Thread 2 completed 617 loops. fork Thread 9 completed 10 loops. fork Thread 10 completed 9 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 18560 loops. alloc Thread 16 completed 44487 loops. alloc Thread 17 completed 87219 loops. lock Thread 21 completed 3769840 loops. lock Thread 22 completed 3037581 loops. lock Thread 23 completed 6097922 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 825 loops. read Thread 1 completed 802 loops. read Thread 2 completed 669 loops. fork Thread 9 completed 9 loops. fork Thread 10 completed 10 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 16877 loops. alloc Thread 16 completed 53277 loops. alloc Thread 17 completed 77986 loops. lock Thread 21 completed 4087218 loops. lock Thread 22 completed 3116357 loops. lock Thread 23 completed 6466214 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 684 loops. read Thread 1 completed 686 loops. read Thread 2 completed 648 loops. fork Thread 9 completed 10 loops. fork Thread 10 completed 9 loops. fork Thread 11 completed 10 loops. alloc Thread 15 completed 19717 loops. alloc Thread 16 completed 47797 loops. alloc Thread 17 completed 78110 loops. lock Thread 21 completed 4580435 loops. lock Thread 22 completed 2974247 loops. lock Thread 23 completed 5843850 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 748 loops. read Thread 1 completed 755 loops. read Thread 2 completed 639 loops. fork Thread 9 completed 9 loops. fork Thread 10 completed 10 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 18037 loops. alloc Thread 16 completed 46293 loops. alloc Thread 17 completed 82560 loops. lock Thread 21 completed 4210008 loops. lock Thread 22 completed 2708008 loops. lock Thread 23 completed 6139723 loops. .. *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0xc4e39a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Coleburn, Randy Sent: Monday, January 13, 2014 3:14 PM To: Tony Hosking Cc: m3devel Subject: Re: [M3devel] CM3 crashing Tony: Yes, it is from the HEAD as of 17 December 2013. Using the @M3paranoidgc flag, On 32-bit Windows XP I get: C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest . *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x431755 = RefSanityCheck + 0x2c in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. On 64-bit Windows 7 I get: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest . *** *** runtime error: *** An array subscript was out of range. *** file "..\src\rw\FileRd.m3", line 83 *** *** *** runtime error: *** An enumeration or subrange value was out of range. *** file "..\src\runtime\common\RTType.m3", line 71 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Tony Hosking [mailto:hosking at cs.purdue.edu] Sent: Monday, January 13, 2014 3:03 PM To: Coleburn, Randy Cc: Peter McKinna; m3devel Subject: EXT:Re: [M3devel] CM3 crashing Is this from the head of source? Also, please run with @M3paranoidgc flag. On Jan 13, 2014, at 2:55 PM, Coleburn, Randy > wrote: I've also tested on 32-bit Windows XP and the thread test program crashes there also. C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386 C:\cm3\Sandbox\cm3\m3-libs>cm3 --version Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2013-12-17 17:28:52 configuration: C:\cm3\bin\cm3.cfg host: NT386 target: NT386 Here is the output: C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Coleburn, Randy Sent: Monday, January 13, 2014 2:17 PM To: Tony Hosking; Peter McKinna Cc: m3devel Subject: Re: [M3devel] CM3 crashing Tony et al: The thread test program fails for me on 64-bit Windows 7. I've listed output from a couple of runs below: C:\cm3\Sandbox>cm3 --version Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2014-01-07 06:55:14 configuration: C:\cm3\bin\cm3.cfg host: NT386 target: NT386 FIRST RUN: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\runtime\common\RTCollector.m3", line 418 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** ...this last assertion repeats until you press CTRL-C to abort the program... SECOND RUN: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) .... *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x776f22d2 *** Stack trace: FP PC Procedure --------- --------- ------------------------------- 0xa6f9a8 0xc6afeb SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3 0xa6f9d0 0x776f22d2 0xa6f9e8 0xc4a39b LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3 0xa6fa10 0xc299f8 GetChar + 0x28 in ..\src\rw\Rd.m3 0xa6fb48 0xc213a8 RApply + 0x1b8 in ..\src\Main.m3 0xa6fb84 0xc4b71b ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3 0xa6fb90 0x76db336a 0xa6fbd0 0x7770bf32 ......... ......... ... more frames ... Maybe this info will be useful in tracking down the problem. --Randy From: Tony Hosking [mailto:hosking at cs.purdue.edu] Sent: Monday, January 13, 2014 11:04 AM To: Peter McKinna Cc: m3devel Subject: EXT:Re: [M3devel] CM3 crashing Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?). In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem. As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC. Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning. Can someone confirm? If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed. On Jan 12, 2014, at 10:54 PM, Peter McKinna > wrote: Hey, I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped the second call to p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one. It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. Regards Peter On Mon, Jan 13, 2014 at 11:25 AM, > wrote: Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3). I figured there aren't actually implementations of Word.Xor, Word.Or, and Word.And as procedures but they get inlined somehow? BTW, any idea about what's wrong with pthreads? Do you think the issue is with the garbage collector or with the threads, off the top of your head? Tony Hosking writes: >Are you saying passing these used to work in PM3? >Sounds like a front-end bug. I=92m curious what changed to break it. > >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: > >>=20 >> The code is: >>=20 >> HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Xor), ab) >> | >> HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Or), ab) >> | >> HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, = >Word.And), ab) >>=20 >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20 > > > >Antony Hosking | Associate Professor | Computer Science | Purdue = >University >305 N. University Street | West Lafayette | IN 47907 | USA >Mobile +1 765 427 5484 > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Jan 14 00:22:12 2014 From: jay.krell at cornell.edu (Jay K) Date: Mon, 13 Jan 2014 23:22:12 +0000 Subject: [M3devel] CM3 crashing In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D3DF@ATLEX04-SRV.SCIRES.LOCAL> References: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D28A@ATLEX04-SRV.SCIRES.LOCAL>, <0BB8FA59C2932741A3A2941A8B9D8BFF9251D3DF@ATLEX04-SRV.SCIRES.LOCAL> Message-ID: I am a bit nervous about our use GetThreadContext in 32bit code on 64bit Windows per here: http://zachsaw.blogspot.com/2010/11/wow64-bug-getthreadcontext-may-return.html And disappointed. Cooperative suspend will fix this someday. - Jay From: rcolebur at SCIRES.COM To: hosking at cs.purdue.edu Date: Mon, 13 Jan 2014 21:02:52 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] CM3 crashing Tony: Here are a couple more runs on 64-bit Windows 7 that yield different results, albeit using the ?-verbose? option. These may give some more clues. I note that in both cases it appears a read thread and a fork thread get the same ID (0) which looks very suspicious. By my read of the code (WITH id = i * nPer + j), I think the 2nd fork thread should be ID=10, not ID=0. Oops, now I see the problem: there is a bug in the PutInt procedure. The test (IF c > 10) should read instead (IF c >= 10). I?ll commit a fix for that momentarily. But this fix just solves the output problem; it doesn?t affect the fact that the program is misbehaving and crashing. RUN #1: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc -verbose Writing file...done Creating read threads... read=0 read=1 read=2 done Creating fork threads... fork=9 fork=0 fork=11 done Creating alloc threads... alloc=15 alloc=16 alloc=17 done Creating lock threads... lock=21 lock=22 lock=23 done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0xc286e0 = Init + 0x79 in ..\src\rw\FileRd.m3 *** Stack trace: FP PC Procedure --------- --------- ------------------------------- 0x7cfd60 0xc6afeb SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3 0x7cfda8 0xc286e0 Init + 0x79 in ..\src\rw\FileRd.m3 0x7cfdd4 0xc2865d Open + 0x4d in ..\src\rw\FileRd.m3 0x7cff0c 0xc21350 RApply + 0x160 in ..\src\Main.m3 0x7cff48 0xc4b71b ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3 0x7cff54 0x76db336a 0x7cff94 0x7770bf32 ......... ......... ... more frames ... RUN #2: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe -verbose Writing file...done Creating read threads... read=0 read=1 read=2 done Creating fork threads... fork=9 fork=0 fork=11 done Creating alloc threads... alloc=15 alloc=16 alloc=17 done Creating lock threads... lock=21 lock=22 lock=23 done running...printing oldest/median age/newest ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 534 loops. read Thread 1 completed 606 loops. read Thread 2 completed 398 loops. fork Thread 9 completed 8 loops. fork Thread 10 completed 8 loops. fork Thread 11 completed 8 loops. alloc Thread 15 completed 18296 loops. alloc Thread 16 completed 44871 loops. alloc Thread 17 completed 79125 loops. lock Thread 21 completed 3845080 loops. lock Thread 22 completed 2537613 loops. lock Thread 23 completed 4506151 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 790 loops. read Thread 1 completed 786 loops. read Thread 2 completed 675 loops. fork Thread 9 completed 9 loops. fork Thread 10 completed 9 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 22091 loops. alloc Thread 16 completed 47532 loops. alloc Thread 17 completed 68901 loops. lock Thread 21 completed 4560731 loops. lock Thread 22 completed 3440795 loops. lock Thread 23 completed 6433538 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 725 loops. read Thread 1 completed 705 loops. read Thread 2 completed 617 loops. fork Thread 9 completed 10 loops. fork Thread 10 completed 9 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 18560 loops. alloc Thread 16 completed 44487 loops. alloc Thread 17 completed 87219 loops. lock Thread 21 completed 3769840 loops. lock Thread 22 completed 3037581 loops. lock Thread 23 completed 6097922 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 825 loops. read Thread 1 completed 802 loops. read Thread 2 completed 669 loops. fork Thread 9 completed 9 loops. fork Thread 10 completed 10 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 16877 loops. alloc Thread 16 completed 53277 loops. alloc Thread 17 completed 77986 loops. lock Thread 21 completed 4087218 loops. lock Thread 22 completed 3116357 loops. lock Thread 23 completed 6466214 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 684 loops. read Thread 1 completed 686 loops. read Thread 2 completed 648 loops. fork Thread 9 completed 10 loops. fork Thread 10 completed 9 loops. fork Thread 11 completed 10 loops. alloc Thread 15 completed 19717 loops. alloc Thread 16 completed 47797 loops. alloc Thread 17 completed 78110 loops. lock Thread 21 completed 4580435 loops. lock Thread 22 completed 2974247 loops. lock Thread 23 completed 5843850 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 748 loops. read Thread 1 completed 755 loops. read Thread 2 completed 639 loops. fork Thread 9 completed 9 loops. fork Thread 10 completed 10 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 18037 loops. alloc Thread 16 completed 46293 loops. alloc Thread 17 completed 82560 loops. lock Thread 21 completed 4210008 loops. lock Thread 22 completed 2708008 loops. lock Thread 23 completed 6139723 loops. .. *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0xc4e39a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Coleburn, Randy Sent: Monday, January 13, 2014 3:14 PM To: Tony Hosking Cc: m3devel Subject: Re: [M3devel] CM3 crashing Tony: Yes, it is from the HEAD as of 17 December 2013. Using the @M3paranoidgc flag, On 32-bit Windows XP I get: C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest . *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x431755 = RefSanityCheck + 0x2c in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. On 64-bit Windows 7 I get: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest . *** *** runtime error: *** An array subscript was out of range. *** file "..\src\rw\FileRd.m3", line 83 *** *** *** runtime error: *** An enumeration or subrange value was out of range. *** file "..\src\runtime\common\RTType.m3", line 71 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Tony Hosking [mailto:hosking at cs.purdue.edu] Sent: Monday, January 13, 2014 3:03 PM To: Coleburn, Randy Cc: Peter McKinna; m3devel Subject: EXT:Re: [M3devel] CM3 crashing Is this from the head of source? Also, please run with @M3paranoidgc flag. On Jan 13, 2014, at 2:55 PM, Coleburn, Randy wrote: I?ve also tested on 32-bit Windows XP and the thread test program crashes there also. C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386 C:\cm3\Sandbox\cm3\m3-libs>cm3 --version Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2013-12-17 17:28:52 configuration: C:\cm3\bin\cm3.cfg host: NT386 target: NT386 Here is the output: C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Coleburn, Randy Sent: Monday, January 13, 2014 2:17 PM To: Tony Hosking; Peter McKinna Cc: m3devel Subject: Re: [M3devel] CM3 crashing Tony et al: The thread test program fails for me on 64-bit Windows 7. I?ve listed output from a couple of runs below: C:\cm3\Sandbox>cm3 --version Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2014-01-07 06:55:14 configuration: C:\cm3\bin\cm3.cfg host: NT386 target: NT386 FIRST RUN: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\runtime\common\RTCollector.m3", line 418 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** ?this last assertion repeats until you press CTRL-C to abort the program? SECOND RUN: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) .... *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x776f22d2 *** Stack trace: FP PC Procedure --------- --------- ------------------------------- 0xa6f9a8 0xc6afeb SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3 0xa6f9d0 0x776f22d2 0xa6f9e8 0xc4a39b LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3 0xa6fa10 0xc299f8 GetChar + 0x28 in ..\src\rw\Rd.m3 0xa6fb48 0xc213a8 RApply + 0x1b8 in ..\src\Main.m3 0xa6fb84 0xc4b71b ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3 0xa6fb90 0x76db336a 0xa6fbd0 0x7770bf32 ......... ......... ... more frames ... Maybe this info will be useful in tracking down the problem. --Randy From: Tony Hosking [mailto:hosking at cs.purdue.edu] Sent: Monday, January 13, 2014 11:04 AM To: Peter McKinna Cc: m3devel Subject: EXT:Re: [M3devel] CM3 crashing Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?). In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem. As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC. Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning. Can someone confirm? If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed. On Jan 12, 2014, at 10:54 PM, Peter McKinna wrote: Hey, I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped the second call to p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one. It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. Regards Peter On Mon, Jan 13, 2014 at 11:25 AM, wrote: Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3). I figured there aren't actually implementations of Word.Xor, Word.Or, and Word.And as procedures but they get inlined somehow? BTW, any idea about what's wrong with pthreads? Do you think the issue is with the garbage collector or with the threads, off the top of your head? Tony Hosking writes: >Are you saying passing these used to work in PM3? >Sounds like a front-end bug. I=92m curious what changed to break it. > >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: > >>=20 >> The code is: >>=20 >> HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Xor), ab) >> | >> HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Or), ab) >> | >> HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, = >Word.And), ab) >>=20 >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20 > > > >Antony Hosking | Associate Professor | Computer Science | Purdue = >University >305 N. University Street | West Lafayette | IN 47907 | USA >Mobile +1 765 427 5484 > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcolebur at SCIRES.COM Tue Jan 14 01:11:07 2014 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Tue, 14 Jan 2014 00:11:07 +0000 Subject: [M3devel] CM3 crashing Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D74F@ATLEX04-SRV.SCIRES.LOCAL> Jay: The article you reference is dated 13 Nov 2010. Do you know whether Microsoft has done anything about this problem over the past 3 years? Also, the thread test program crashes on 32-bit XP, so there must be some additional bug(s) somewhere besides the WOW64 bug wrt GetThreadContext. --Randy From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Monday, January 13, 2014 6:22 PM To: Coleburn, Randy; Tony Cc: m3devel Subject: EXT:RE: [M3devel] CM3 crashing I am a bit nervous about our use GetThreadContext in 32bit code on 64bit Windows per here: http://zachsaw.blogspot.com/2010/11/wow64-bug-getthreadcontext-may-return.html And disappointed. Cooperative suspend will fix this someday. - Jay ________________________________ From: rcolebur at SCIRES.COM To: hosking at cs.purdue.edu Date: Mon, 13 Jan 2014 21:02:52 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] CM3 crashing Tony: Here are a couple more runs on 64-bit Windows 7 that yield different results, albeit using the "-verbose" option. These may give some more clues. I note that in both cases it appears a read thread and a fork thread get the same ID (0) which looks very suspicious. By my read of the code (WITH id = i * nPer + j), I think the 2nd fork thread should be ID=10, not ID=0. Oops, now I see the problem: there is a bug in the PutInt procedure. The test (IF c > 10) should read instead (IF c >= 10). I'll commit a fix for that momentarily. But this fix just solves the output problem; it doesn't affect the fact that the program is misbehaving and crashing. RUN #1: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc -verbose Writing file...done Creating read threads... read=0 read=1 read=2 done Creating fork threads... fork=9 fork=0 fork=11 done Creating alloc threads... alloc=15 alloc=16 alloc=17 done Creating lock threads... lock=21 lock=22 lock=23 done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0xc286e0 = Init + 0x79 in ..\src\rw\FileRd.m3 *** Stack trace: FP PC Procedure --------- --------- ------------------------------- 0x7cfd60 0xc6afeb SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3 0x7cfda8 0xc286e0 Init + 0x79 in ..\src\rw\FileRd.m3 0x7cfdd4 0xc2865d Open + 0x4d in ..\src\rw\FileRd.m3 0x7cff0c 0xc21350 RApply + 0x160 in ..\src\Main.m3 0x7cff48 0xc4b71b ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3 0x7cff54 0x76db336a 0x7cff94 0x7770bf32 ......... ......... ... more frames ... RUN #2: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe -verbose Writing file...done Creating read threads... read=0 read=1 read=2 done Creating fork threads... fork=9 fork=0 fork=11 done Creating alloc threads... alloc=15 alloc=16 alloc=17 done Creating lock threads... lock=21 lock=22 lock=23 done running...printing oldest/median age/newest ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 534 loops. read Thread 1 completed 606 loops. read Thread 2 completed 398 loops. fork Thread 9 completed 8 loops. fork Thread 10 completed 8 loops. fork Thread 11 completed 8 loops. alloc Thread 15 completed 18296 loops. alloc Thread 16 completed 44871 loops. alloc Thread 17 completed 79125 loops. lock Thread 21 completed 3845080 loops. lock Thread 22 completed 2537613 loops. lock Thread 23 completed 4506151 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 790 loops. read Thread 1 completed 786 loops. read Thread 2 completed 675 loops. fork Thread 9 completed 9 loops. fork Thread 10 completed 9 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 22091 loops. alloc Thread 16 completed 47532 loops. alloc Thread 17 completed 68901 loops. lock Thread 21 completed 4560731 loops. lock Thread 22 completed 3440795 loops. lock Thread 23 completed 6433538 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 725 loops. read Thread 1 completed 705 loops. read Thread 2 completed 617 loops. fork Thread 9 completed 10 loops. fork Thread 10 completed 9 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 18560 loops. alloc Thread 16 completed 44487 loops. alloc Thread 17 completed 87219 loops. lock Thread 21 completed 3769840 loops. lock Thread 22 completed 3037581 loops. lock Thread 23 completed 6097922 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 825 loops. read Thread 1 completed 802 loops. read Thread 2 completed 669 loops. fork Thread 9 completed 9 loops. fork Thread 10 completed 10 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 16877 loops. alloc Thread 16 completed 53277 loops. alloc Thread 17 completed 77986 loops. lock Thread 21 completed 4087218 loops. lock Thread 22 completed 3116357 loops. lock Thread 23 completed 6466214 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 684 loops. read Thread 1 completed 686 loops. read Thread 2 completed 648 loops. fork Thread 9 completed 10 loops. fork Thread 10 completed 9 loops. fork Thread 11 completed 10 loops. alloc Thread 15 completed 19717 loops. alloc Thread 16 completed 47797 loops. alloc Thread 17 completed 78110 loops. lock Thread 21 completed 4580435 loops. lock Thread 22 completed 2974247 loops. lock Thread 23 completed 5843850 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 748 loops. read Thread 1 completed 755 loops. read Thread 2 completed 639 loops. fork Thread 9 completed 9 loops. fork Thread 10 completed 10 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 18037 loops. alloc Thread 16 completed 46293 loops. alloc Thread 17 completed 82560 loops. lock Thread 21 completed 4210008 loops. lock Thread 22 completed 2708008 loops. lock Thread 23 completed 6139723 loops. .. *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0xc4e39a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Coleburn, Randy Sent: Monday, January 13, 2014 3:14 PM To: Tony Hosking Cc: m3devel Subject: Re: [M3devel] CM3 crashing Tony: Yes, it is from the HEAD as of 17 December 2013. Using the @M3paranoidgc flag, On 32-bit Windows XP I get: C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest . *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x431755 = RefSanityCheck + 0x2c in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. On 64-bit Windows 7 I get: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest . *** *** runtime error: *** An array subscript was out of range. *** file "..\src\rw\FileRd.m3", line 83 *** *** *** runtime error: *** An enumeration or subrange value was out of range. *** file "..\src\runtime\common\RTType.m3", line 71 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Tony Hosking [mailto:hosking at cs.purdue.edu] Sent: Monday, January 13, 2014 3:03 PM To: Coleburn, Randy Cc: Peter McKinna; m3devel Subject: EXT:Re: [M3devel] CM3 crashing Is this from the head of source? Also, please run with @M3paranoidgc flag. On Jan 13, 2014, at 2:55 PM, Coleburn, Randy > wrote: I've also tested on 32-bit Windows XP and the thread test program crashes there also. C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386 C:\cm3\Sandbox\cm3\m3-libs>cm3 --version Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2013-12-17 17:28:52 configuration: C:\cm3\bin\cm3.cfg host: NT386 target: NT386 Here is the output: C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Coleburn, Randy Sent: Monday, January 13, 2014 2:17 PM To: Tony Hosking; Peter McKinna Cc: m3devel Subject: Re: [M3devel] CM3 crashing Tony et al: The thread test program fails for me on 64-bit Windows 7. I've listed output from a couple of runs below: C:\cm3\Sandbox>cm3 --version Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2014-01-07 06:55:14 configuration: C:\cm3\bin\cm3.cfg host: NT386 target: NT386 FIRST RUN: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\runtime\common\RTCollector.m3", line 418 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** ...this last assertion repeats until you press CTRL-C to abort the program... SECOND RUN: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) .... *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x776f22d2 *** Stack trace: FP PC Procedure --------- --------- ------------------------------- 0xa6f9a8 0xc6afeb SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3 0xa6f9d0 0x776f22d2 0xa6f9e8 0xc4a39b LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3 0xa6fa10 0xc299f8 GetChar + 0x28 in ..\src\rw\Rd.m3 0xa6fb48 0xc213a8 RApply + 0x1b8 in ..\src\Main.m3 0xa6fb84 0xc4b71b ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3 0xa6fb90 0x76db336a 0xa6fbd0 0x7770bf32 ......... ......... ... more frames ... Maybe this info will be useful in tracking down the problem. --Randy From: Tony Hosking [mailto:hosking at cs.purdue.edu] Sent: Monday, January 13, 2014 11:04 AM To: Peter McKinna Cc: m3devel Subject: EXT:Re: [M3devel] CM3 crashing Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?). In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem. As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC. Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning. Can someone confirm? If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed. On Jan 12, 2014, at 10:54 PM, Peter McKinna > wrote: Hey, I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped the second call to p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one. It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. Regards Peter On Mon, Jan 13, 2014 at 11:25 AM, > wrote: Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3). I figured there aren't actually implementations of Word.Xor, Word.Or, and Word.And as procedures but they get inlined somehow? BTW, any idea about what's wrong with pthreads? Do you think the issue is with the garbage collector or with the threads, off the top of your head? Tony Hosking writes: >Are you saying passing these used to work in PM3? >Sounds like a front-end bug. I=92m curious what changed to break it. > >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: > >>=20 >> The code is: >>=20 >> HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Xor), ab) >> | >> HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Or), ab) >> | >> HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, = >Word.And), ab) >>=20 >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20 > > > >Antony Hosking | Associate Professor | Computer Science | Purdue = >University >305 N. University Street | West Lafayette | IN 47907 | USA >Mobile +1 765 427 5484 > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Jan 14 01:21:49 2014 From: jay.krell at cornell.edu (Jay K) Date: Tue, 14 Jan 2014 00:21:49 +0000 Subject: [M3devel] CM3 crashing In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D74F@ATLEX04-SRV.SCIRES.LOCAL> References: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D74F@ATLEX04-SRV.SCIRES.LOCAL> Message-ID: I don't know. We can try writing a stress test for it. It looks like one is available. Agreed -- work out the bugs in the native Windows and Posix systems. - Jay From: rcolebur at SCIRES.COM To: jay.krell at cornell.edu; hosking at cs.purdue.edu Date: Tue, 14 Jan 2014 00:11:07 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] CM3 crashing Jay: The article you reference is dated 13 Nov 2010. Do you know whether Microsoft has done anything about this problem over the past 3 years? Also, the thread test program crashes on 32-bit XP, so there must be some additional bug(s) somewhere besides the WOW64 bug wrt GetThreadContext. --Randy From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Monday, January 13, 2014 6:22 PM To: Coleburn, Randy; Tony Cc: m3devel Subject: EXT:RE: [M3devel] CM3 crashing I am a bit nervous about our use GetThreadContext in 32bit code on 64bit Windows per here: http://zachsaw.blogspot.com/2010/11/wow64-bug-getthreadcontext-may-return.html And disappointed. Cooperative suspend will fix this someday. - Jay From: rcolebur at SCIRES.COM To: hosking at cs.purdue.edu Date: Mon, 13 Jan 2014 21:02:52 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] CM3 crashing Tony: Here are a couple more runs on 64-bit Windows 7 that yield different results, albeit using the ?-verbose? option. These may give some more clues. I note that in both cases it appears a read thread and a fork thread get the same ID (0) which looks very suspicious. By my read of the code (WITH id = i * nPer + j), I think the 2nd fork thread should be ID=10, not ID=0. Oops, now I see the problem: there is a bug in the PutInt procedure. The test (IF c > 10) should read instead (IF c >= 10). I?ll commit a fix for that momentarily. But this fix just solves the output problem; it doesn?t affect the fact that the program is misbehaving and crashing. RUN #1: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc -verbose Writing file...done Creating read threads... read=0 read=1 read=2 done Creating fork threads... fork=9 fork=0 fork=11 done Creating alloc threads... alloc=15 alloc=16 alloc=17 done Creating lock threads... lock=21 lock=22 lock=23 done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0xc286e0 = Init + 0x79 in ..\src\rw\FileRd.m3 *** Stack trace: FP PC Procedure --------- --------- ------------------------------- 0x7cfd60 0xc6afeb SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3 0x7cfda8 0xc286e0 Init + 0x79 in ..\src\rw\FileRd.m3 0x7cfdd4 0xc2865d Open + 0x4d in ..\src\rw\FileRd.m3 0x7cff0c 0xc21350 RApply + 0x160 in ..\src\Main.m3 0x7cff48 0xc4b71b ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3 0x7cff54 0x76db336a 0x7cff94 0x7770bf32 ......... ......... ... more frames ... RUN #2: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe -verbose Writing file...done Creating read threads... read=0 read=1 read=2 done Creating fork threads... fork=9 fork=0 fork=11 done Creating alloc threads... alloc=15 alloc=16 alloc=17 done Creating lock threads... lock=21 lock=22 lock=23 done running...printing oldest/median age/newest ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 534 loops. read Thread 1 completed 606 loops. read Thread 2 completed 398 loops. fork Thread 9 completed 8 loops. fork Thread 10 completed 8 loops. fork Thread 11 completed 8 loops. alloc Thread 15 completed 18296 loops. alloc Thread 16 completed 44871 loops. alloc Thread 17 completed 79125 loops. lock Thread 21 completed 3845080 loops. lock Thread 22 completed 2537613 loops. lock Thread 23 completed 4506151 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 790 loops. read Thread 1 completed 786 loops. read Thread 2 completed 675 loops. fork Thread 9 completed 9 loops. fork Thread 10 completed 9 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 22091 loops. alloc Thread 16 completed 47532 loops. alloc Thread 17 completed 68901 loops. lock Thread 21 completed 4560731 loops. lock Thread 22 completed 3440795 loops. lock Thread 23 completed 6433538 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 725 loops. read Thread 1 completed 705 loops. read Thread 2 completed 617 loops. fork Thread 9 completed 10 loops. fork Thread 10 completed 9 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 18560 loops. alloc Thread 16 completed 44487 loops. alloc Thread 17 completed 87219 loops. lock Thread 21 completed 3769840 loops. lock Thread 22 completed 3037581 loops. lock Thread 23 completed 6097922 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 825 loops. read Thread 1 completed 802 loops. read Thread 2 completed 669 loops. fork Thread 9 completed 9 loops. fork Thread 10 completed 10 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 16877 loops. alloc Thread 16 completed 53277 loops. alloc Thread 17 completed 77986 loops. lock Thread 21 completed 4087218 loops. lock Thread 22 completed 3116357 loops. lock Thread 23 completed 6466214 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 684 loops. read Thread 1 completed 686 loops. read Thread 2 completed 648 loops. fork Thread 9 completed 10 loops. fork Thread 10 completed 9 loops. fork Thread 11 completed 10 loops. alloc Thread 15 completed 19717 loops. alloc Thread 16 completed 47797 loops. alloc Thread 17 completed 78110 loops. lock Thread 21 completed 4580435 loops. lock Thread 22 completed 2974247 loops. lock Thread 23 completed 5843850 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 748 loops. read Thread 1 completed 755 loops. read Thread 2 completed 639 loops. fork Thread 9 completed 9 loops. fork Thread 10 completed 10 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 18037 loops. alloc Thread 16 completed 46293 loops. alloc Thread 17 completed 82560 loops. lock Thread 21 completed 4210008 loops. lock Thread 22 completed 2708008 loops. lock Thread 23 completed 6139723 loops. .. *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0xc4e39a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Coleburn, Randy Sent: Monday, January 13, 2014 3:14 PM To: Tony Hosking Cc: m3devel Subject: Re: [M3devel] CM3 crashing Tony: Yes, it is from the HEAD as of 17 December 2013. Using the @M3paranoidgc flag, On 32-bit Windows XP I get: C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest . *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x431755 = RefSanityCheck + 0x2c in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. On 64-bit Windows 7 I get: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest . *** *** runtime error: *** An array subscript was out of range. *** file "..\src\rw\FileRd.m3", line 83 *** *** *** runtime error: *** An enumeration or subrange value was out of range. *** file "..\src\runtime\common\RTType.m3", line 71 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Tony Hosking [mailto:hosking at cs.purdue.edu] Sent: Monday, January 13, 2014 3:03 PM To: Coleburn, Randy Cc: Peter McKinna; m3devel Subject: EXT:Re: [M3devel] CM3 crashing Is this from the head of source? Also, please run with @M3paranoidgc flag. On Jan 13, 2014, at 2:55 PM, Coleburn, Randy wrote: I?ve also tested on 32-bit Windows XP and the thread test program crashes there also. C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386 C:\cm3\Sandbox\cm3\m3-libs>cm3 --version Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2013-12-17 17:28:52 configuration: C:\cm3\bin\cm3.cfg host: NT386 target: NT386 Here is the output: C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Coleburn, Randy Sent: Monday, January 13, 2014 2:17 PM To: Tony Hosking; Peter McKinna Cc: m3devel Subject: Re: [M3devel] CM3 crashing Tony et al: The thread test program fails for me on 64-bit Windows 7. I?ve listed output from a couple of runs below: C:\cm3\Sandbox>cm3 --version Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2014-01-07 06:55:14 configuration: C:\cm3\bin\cm3.cfg host: NT386 target: NT386 FIRST RUN: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\runtime\common\RTCollector.m3", line 418 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** ?this last assertion repeats until you press CTRL-C to abort the program? SECOND RUN: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) .... *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x776f22d2 *** Stack trace: FP PC Procedure --------- --------- ------------------------------- 0xa6f9a8 0xc6afeb SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3 0xa6f9d0 0x776f22d2 0xa6f9e8 0xc4a39b LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3 0xa6fa10 0xc299f8 GetChar + 0x28 in ..\src\rw\Rd.m3 0xa6fb48 0xc213a8 RApply + 0x1b8 in ..\src\Main.m3 0xa6fb84 0xc4b71b ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3 0xa6fb90 0x76db336a 0xa6fbd0 0x7770bf32 ......... ......... ... more frames ... Maybe this info will be useful in tracking down the problem. --Randy From: Tony Hosking [mailto:hosking at cs.purdue.edu] Sent: Monday, January 13, 2014 11:04 AM To: Peter McKinna Cc: m3devel Subject: EXT:Re: [M3devel] CM3 crashing Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?). In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem. As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC. Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning. Can someone confirm? If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed. On Jan 12, 2014, at 10:54 PM, Peter McKinna wrote: Hey, I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped the second call to p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one. It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. Regards Peter On Mon, Jan 13, 2014 at 11:25 AM, wrote: Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3). I figured there aren't actually implementations of Word.Xor, Word.Or, and Word.And as procedures but they get inlined somehow? BTW, any idea about what's wrong with pthreads? Do you think the issue is with the garbage collector or with the threads, off the top of your head? Tony Hosking writes: >Are you saying passing these used to work in PM3? >Sounds like a front-end bug. I=92m curious what changed to break it. > >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: > >>=20 >> The code is: >>=20 >> HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Xor), ab) >> | >> HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Or), ab) >> | >> HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, = >Word.And), ab) >>=20 >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20 > > > >Antony Hosking | Associate Professor | Computer Science | Purdue = >University >305 N. University Street | West Lafayette | IN 47907 | USA >Mobile +1 765 427 5484 > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.mckinna at gmail.com Mon Jan 13 04:54:08 2014 From: peter.mckinna at gmail.com (Peter McKinna) Date: Mon, 13 Jan 2014 14:54:08 +1100 Subject: [M3devel] CM3 crashing In-Reply-To: <20140113002539.EC4EE1A2080@async.async.caltech.edu> References: <20140112175317.9F0031A2080@async.async.caltech.edu> <20140112175830.C51881A2080@async.async.caltech.edu> <20140113002539.EC4EE1A2080@async.async.caltech.edu> Message-ID: Hey, I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped the second call to p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one. It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. Regards Peter On Mon, Jan 13, 2014 at 11:25 AM, wrote: > Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a > reason to switch to CM3). > > I figured there aren't actually implementations of Word.Xor, Word.Or, > and Word.And as procedures but they get inlined somehow? > > BTW, any idea about what's wrong with pthreads? Do you think the issue > is with the garbage collector or with the threads, off the top of your > head? > > Tony Hosking writes: > >Are you saying passing these used to work in PM3? > >Sounds like a front-end bug. I=92m curious what changed to break it. > > > >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: > > > >>=20 > >> The code is: > >>=20 > >> HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, = > >Word.Xor), ab) > >> | > >> HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, = > >Word.Or), ab) > >> | > >> HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, = > >Word.And), ab) > >>=20 > >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20 > > > > > > > >Antony Hosking | Associate Professor | Computer Science | Purdue = > >University > >305 N. University Street | West Lafayette | IN 47907 | USA > >Mobile +1 765 427 5484 > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jan 15 06:01:16 2014 From: jay.krell at cornell.edu (Jay K) Date: Wed, 15 Jan 2014 05:01:16 +0000 Subject: [M3devel] CM3 crashing In-Reply-To: References: <20140112175317.9F0031A2080@async.async.caltech.edu>, <20140112175830.C51881A2080@async.async.caltech.edu>, , <20140113002539.EC4EE1A2080@async.async.caltech.edu>, Message-ID: Here is my failed attempt to demonstrate the wow64 bug. Make sense? Repros for anyone? I only tried Windows 8 so far. #undef NDEBUG #include #include #include void* _AddressOfReturnAddress(void); volatile size_t expected; #define PAGE 4096 __declspec(noinline) void X(void) { expected = (size_t)_AddressOfReturnAddress(); Sleep(1); expected = 0; } __declspec(noinline) void F1(void) { volatile char a[PAGE * 8]; X(); } __declspec(noinline) void F2(void) { volatile char a[PAGE * 4]; X(); } __declspec(noinline) unsigned long __stdcall Thread(PVOID parameter) { while (1) { F1(); F2(); } return 0; } int __cdecl wmain() { unsigned __int64 count = 0; HANDLE thread = CreateThread(0, 0, Thread, 0, 0, 0); CONTEXT context; ZeroMemory(&context, sizeof(context)); context.ContextFlags = CONTEXT_CONTROL; //while (count++ < 1000000) while (1) { SuspendThread(thread); GetThreadContext(thread, &context); assert(expected == 0 || (context.Esp && context.Esp < expected)); ResumeThread(thread); } } - Jay Date: Mon, 13 Jan 2014 14:54:08 +1100 From: peter.mckinna at gmail.com To: mika at async.caltech.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] CM3 crashing Hey, I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped the second call to p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one. It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. Regards Peter On Mon, Jan 13, 2014 at 11:25 AM, wrote: Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3). I figured there aren't actually implementations of Word.Xor, Word.Or, and Word.And as procedures but they get inlined somehow? BTW, any idea about what's wrong with pthreads? Do you think the issue is with the garbage collector or with the threads, off the top of your head? Tony Hosking writes: >Are you saying passing these used to work in PM3? >Sounds like a front-end bug. I=92m curious what changed to break it. > >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: > >>=20 >> The code is: >>=20 >> HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Xor), ab) >> | >> HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Or), ab) >> | >> HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, = >Word.And), ab) >>=20 >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20 > > > >Antony Hosking | Associate Professor | Computer Science | Purdue = >University >305 N. University Street | West Lafayette | IN 47907 | USA >Mobile +1 765 427 5484 > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jan 15 07:48:30 2014 From: jay.krell at cornell.edu (Jay K) Date: Wed, 15 Jan 2014 06:48:30 +0000 Subject: [M3devel] CM3 crashing In-Reply-To: References: <20140112175317.9F0031A2080@async.async.caltech.edu>, , <20140112175830.C51881A2080@async.async.caltech.edu>, , , , <20140113002539.EC4EE1A2080@async.async.caltech.edu>, , , Message-ID: ok, I augmented the test a bit and commited it to CVS and I can see it fail now. Does the test make sense to people? Thanks, - Jay From: jay.krell at cornell.edu To: peter.mckinna at gmail.com; mika at async.caltech.edu Date: Wed, 15 Jan 2014 05:01:16 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] CM3 crashing Here is my failed attempt to demonstrate the wow64 bug. Make sense? Repros for anyone? I only tried Windows 8 so far. #undef NDEBUG #include #include #include void* _AddressOfReturnAddress(void); volatile size_t expected; #define PAGE 4096 __declspec(noinline) void X(void) { expected = (size_t)_AddressOfReturnAddress(); Sleep(1); expected = 0; } __declspec(noinline) void F1(void) { volatile char a[PAGE * 8]; X(); } __declspec(noinline) void F2(void) { volatile char a[PAGE * 4]; X(); } __declspec(noinline) unsigned long __stdcall Thread(PVOID parameter) { while (1) { F1(); F2(); } return 0; } int __cdecl wmain() { unsigned __int64 count = 0; HANDLE thread = CreateThread(0, 0, Thread, 0, 0, 0); CONTEXT context; ZeroMemory(&context, sizeof(context)); context.ContextFlags = CONTEXT_CONTROL; //while (count++ < 1000000) while (1) { SuspendThread(thread); GetThreadContext(thread, &context); assert(expected == 0 || (context.Esp && context.Esp < expected)); ResumeThread(thread); } } - Jay Date: Mon, 13 Jan 2014 14:54:08 +1100 From: peter.mckinna at gmail.com To: mika at async.caltech.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] CM3 crashing Hey, I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped the second call to p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one. It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. Regards Peter On Mon, Jan 13, 2014 at 11:25 AM, wrote: Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3). I figured there aren't actually implementations of Word.Xor, Word.Or, and Word.And as procedures but they get inlined somehow? BTW, any idea about what's wrong with pthreads? Do you think the issue is with the garbage collector or with the threads, off the top of your head? Tony Hosking writes: >Are you saying passing these used to work in PM3? >Sounds like a front-end bug. I=92m curious what changed to break it. > >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: > >>=20 >> The code is: >>=20 >> HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Xor), ab) >> | >> HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Or), ab) >> | >> HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, = >Word.And), ab) >>=20 >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20 > > > >Antony Hosking | Associate Professor | Computer Science | Purdue = >University >305 N. University Street | West Lafayette | IN 47907 | USA >Mobile +1 765 427 5484 > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sat Jan 18 07:22:53 2014 From: jay.krell at cornell.edu (Jay K) Date: Sat, 18 Jan 2014 06:22:53 +0000 Subject: [M3devel] Win32 SuspendThread? Message-ID: This program also doesn't behave as expected, native, nothing to do with wow64. Anyone else please confirm: 1) my expectations -- it should never print anything 2) their importance -- garbage collector depends on it 3) their not being met -- stuff gets printed This is in the CVS repository, scratch/wow64stack/sync2.cpp I am following up further. Maybe we should get cooperative suspend really going? Thank you. - Jay #include #include volatile long value; unsigned long __stdcall Thread(PVOID parameter) { while (1) InterlockedIncrement(&value); return 0; } int __cdecl main() { HANDLE thread = CreateThread(0, 0, Thread, 0, 0, 0); UINT i = 0; while (1) { i += 1; if (SuspendThread(thread) == (DWORD)-1) { printf("suspend failed %X\n", GetLastError()); Sleep(1); continue; } volatile long a = value; volatile long b = value; if (a != b) { printf("%d %d %d %d\n", i, a, b, b - a); } ResumeThread(thread); } } -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcolebur at SCIRES.COM Sat Jan 18 08:11:45 2014 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Sat, 18 Jan 2014 07:11:45 +0000 Subject: [M3devel] Win32 SuspendThread? Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL> Jay: I'm very concerned about the threading not working properly on both 32-bit and 64-bit Windows. The Thread Test program crashes for me on both platforms. I haven't tried your new test program yet. --Randy From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Saturday, January 18, 2014 1:23 AM To: m3devel Subject: EXT:[M3devel] Win32 SuspendThread? This program also doesn't behave as expected, native, nothing to do with wow64. Anyone else please confirm: 1) my expectations -- it should never print anything 2) their importance -- garbage collector depends on it 3) their not being met -- stuff gets printed This is in the CVS repository, scratch/wow64stack/sync2.cpp I am following up further. Maybe we should get cooperative suspend really going? Thank you. - Jay #include #include volatile long value; unsigned long __stdcall Thread(PVOID parameter) { while (1) InterlockedIncrement(&value); return 0; } int __cdecl main() { HANDLE thread = CreateThread(0, 0, Thread, 0, 0, 0); UINT i = 0; while (1) { i += 1; if (SuspendThread(thread) == (DWORD)-1) { printf("suspend failed %X\n", GetLastError()); Sleep(1); continue; } volatile long a = value; volatile long b = value; if (a != b) { printf("%d %d %d %d\n", i, a, b, b - a); } ResumeThread(thread); } } -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Sat Jan 18 09:24:11 2014 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 18 Jan 2014 09:24:11 +0100 Subject: [M3devel] Cooperative suspend? Message-ID: <8BC1682A-3760-44D0-BA67-3A0F3EA10E27@m3w.org> Jay, What exactly do you mean? Isn?t it cooperative already? Thread stopping the world informs others about its intent, and they suspend themselves on a semaphore. Windows implementation of threading does forced suspend? dd -- Dragi?a Duri? dragisha at m3w.org -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Sun Jan 19 04:40:06 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sat, 18 Jan 2014 19:40:06 -0800 Subject: [M3devel] how is this possible? Message-ID: <20140119034006.34F511A208F@async.async.caltech.edu> Any hypothesis is welcome... (311)truffles:~/t/btc/hparse/src>../AMD64_LINUX/hparse and.scm *** *** runtime error: *** An explicit or implicit NARROW() operation failed. *** file "../AMD64_LINUX/BDDSchemeStubs.m3", line 619 *** Abort (312)truffles:~/t/btc/hparse/src> 613 PROCEDURE ToModula_BDD_T(x : SchemeObject.T) : (BDD.T) RAISES { Scheme.E } = 614 BEGIN 615 IF x # NIL AND ISTYPE(x, REF BDD.T) THEN RETURN NARROW(x,REF BDD.T)^ END; 616 IF NOT ISTYPE(x,BDD.T) THEN 617 RAISE Scheme.E("Not of type BDD.T : " & SchemeUtils.Stringify(x)) 618 END; 619 RETURN x 620 END ToModula_BDD_T; The object is of type BDD.Root (I checked the TYPECODE), which is declared thus: TYPE Root = T BRANDED Brand & " Root" OBJECT mu : MUTEX; (* as yet unused *) id : CARDINAL; tab : BDDTripleHash.T; cache : ARRAY Op OF BDDTripleHash.T; END; SchemeObject.T is the same as REFANY I'm not even sure where to start debugging this. The code "should work"... Indeed: > (rttype-typecode a) 144 > (RTBrand.GetByTC 144) "BDD 0.1 Root" > (RTBrand.GetByTC 143) "BDD 0.1" > (RTType.Supertype 143) 4 (4 is REFANY) > (BDD.And a a) EXCEPTION! RuntimeError! An explicit or implicit NARROW() operation failed. > a-and-a > (rttype-typecode a-and-a) 144 Mika From jay.krell at cornell.edu Sun Jan 19 02:02:56 2014 From: jay.krell at cornell.edu (Jay) Date: Sat, 18 Jan 2014 17:02:56 -0800 Subject: [M3devel] Win32 SuspendThread? In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL> References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL> Message-ID: <9EC3F463-F248-4C79-AF3F-130D2252B8B0@gmail.com> For native I'm less worried now. For wow64 I'm still worried & hope to follow up further. - Jay On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" wrote: > Jay: > > I?m very concerned about the threading not working properly on both 32-bit and 64-bit Windows. > > The Thread Test program crashes for me on both platforms. > > I haven?t tried your new test program yet. > > --Randy > > From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K > Sent: Saturday, January 18, 2014 1:23 AM > To: m3devel > Subject: EXT:[M3devel] Win32 SuspendThread? > > This program also doesn't behave as expected, native, nothing to do with wow64. > Anyone else please confirm: > 1) my expectations -- it should never print anything > 2) their importance -- garbage collector depends on it > 3) their not being met -- stuff gets printed > This is in the CVS repository, scratch/wow64stack/sync2.cpp > I am following up further. > Maybe we should get cooperative suspend really going? > Thank you. > - Jay > > #include > #include > volatile long value; > unsigned long __stdcall Thread(PVOID parameter) > { > while (1) > InterlockedIncrement(&value); > return 0; > } > int __cdecl main() > { > HANDLE thread = CreateThread(0, 0, Thread, 0, 0, 0); > UINT i = 0; > while (1) > { > i += 1; > if (SuspendThread(thread) == (DWORD)-1) > { > printf("suspend failed %X\n", GetLastError()); > Sleep(1); > continue; > } > volatile long a = value; > volatile long b = value; > if (a != b) > { > printf("%d %d %d %d\n", i, a, b, b - a); > } > ResumeThread(thread); > } > } -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Sun Jan 19 10:41:39 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sun, 19 Jan 2014 01:41:39 -0800 Subject: [M3devel] something wrong with ISTYPE ... Message-ID: <20140119094139.3917E1A208F@async.async.caltech.edu> Hi m3devel, There is definitely something wrong with ISTYPE. The problem I reported earlier goes away if I change the code like so: (416)truffles:~/t/btc/hparse/AMD64_LINUX>diff -c BDDSchemeStubs.m3.old BDDSchemeStubs.m3 *** BDDSchemeStubs.m3.old 2014-01-19 01:13:33.000000000 -0800 --- BDDSchemeStubs.m3 2014-01-19 01:36:52.000000000 -0800 *************** *** 613,622 **** PROCEDURE ToModula_BDD_T(x : SchemeObject.T) : (BDD.T) RAISES { Scheme.E } = BEGIN IF x # NIL AND ISTYPE(x, REF BDD.T) THEN RETURN NARROW(x,REF BDD.T)^ END; ! IF NOT ISTYPE(x,BDD.T) THEN RAISE Scheme.E("Not of type BDD.T : " & SchemeUtils.Stringify(x)) ! END; ! RETURN x END ToModula_BDD_T; --- 613,624 ---- PROCEDURE ToModula_BDD_T(x : SchemeObject.T) : (BDD.T) RAISES { Scheme.E } = BEGIN IF x # NIL AND ISTYPE(x, REF BDD.T) THEN RETURN NARROW(x,REF BDD.T)^ END; ! TYPECASE x OF ! BDD.T(xx) => RETURN xx ! ELSE RAISE Scheme.E("Not of type BDD.T : " & SchemeUtils.Stringify(x)) ! END ! END ToModula_BDD_T; This shouldn't be possible... it's the same object as before, with the same return type. I don't get an exception (I shouldn't either)---the return just behaves normally and the program works. Why doesn't it work with ISTYPE but only with TYPECASE? BTW, this is an old bug. Same behavior with ancient PM3. Where to look in the code? Mika From adacore at marino.st Sun Jan 19 11:03:27 2014 From: adacore at marino.st (John Marino) Date: Sun, 19 Jan 2014 11:03:27 +0100 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) Message-ID: <52DBA2EF.4020101@marino.st> Hi all, I've been trying to port cm3 to DragonFly x86-64. I'm able to build cm3cg on DragonFly, and use it to build libraries (e.g. libm3core) that were targeted for AMD64_FREEBSD. However, when I set the HOST to AMD64_FREEBSD and TARGET to AMD64_DRAGONFLY, I get the following error, "Fatal Error: unrecognized target machine: TARGET = AMD64_DRAGONFLY" (context below) ========================================================= > /work/lang/modula3/work/modula3-5.8.6/scripts/pkgmap.sh -c "/work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS " m3core > ROOT = /work/lang/modula3/work/modula3-5.8.6 > CM3_ROOT = /work/lang/modula3/work/modula3-5.8.6 > M3GDB = yes > M3OSTYPE = POSIX > TARGET = AMD64_DRAGONFLY > GCC_BACKEND = yes > INSTALLROOT = /work/lang/modula3/work/stage/usr/local > PKGSDB = /work/lang/modula3/work/modula3-5.8.6/scripts/PKGS > GREP = egrep > GMAKE = gmake > TMPDIR = /var/tmp > EXE = > SL = / > TAR = tar > CM3ROOT = /work/lang/modula3/work/modula3-5.8.6 > CM3VERSION = 5.8.6 > CM3VERSIONNUM = 050806 > CM3LASTCHANGED = 2010-04-11 > FIND = /usr/bin/find > EGREP = egrep > === package m3-libs/m3core === > +++ /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS +++ > --- building in AMD64_DRAGONFLY --- > > ignoring ../src/m3overrides > > > Fatal Error: unrecognized target machine: TARGET = AMD64_DRAGONFLY > > *** execution of /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS failed *** > *** [assembler-code] Error code 1 ========================================================= I thought I added all the target definitions but apparently I've missed something. I also realize the current repo looks a lot different, but I wanted to use the last official release. I've attached a concatenated file of patches showing what I've changed. Could somebody suggest what data is missing that causes the error above? Regards, John -------------- next part -------------- --- m3-libs/libm3/src/os/POSIX/m3makefile.orig 2009-07-24 10:02:40.000000000 +0000 +++ m3-libs/libm3/src/os/POSIX/m3makefile @@ -84,6 +84,7 @@ local readonly DefaultOSName = { "MIPS64_OPENBSD" : "OpenBSD", "SPARC64_SOLARIS" : "Solaris", "I386_OPENBSD" : "OpenBSD", + "AMD64_DRAGONFLY" : "DragonFly", "AMD64_FREEBSD" : "FreeBSD", "PA32_HPUX" : "HPUX", "PA64_HPUX" : "HPUX", @@ -92,6 +93,7 @@ local readonly DefaultOSName = { "AIX" : "AIX", "CYGWIN" : "Cygwin", "DARWIN" : "Darwin", + "DRAGONFLY" : "DragonFly", "FREEBSD" : "FreeBSD", "HPUX" : "HPUX", "INTERIX" : "Interix", @@ -156,6 +158,7 @@ local readonly DefaultArch = { "SPARC64_SOLARIS" : "sparc64", "I386_OPENBSD" : "i686", "AMD64_FREEBSD" : "AMD64", + "AMD64_DRAGONFLY" : "AMD64", "PA32_HPUX" : "hppa", "PA64_HPUX" : "hppa64", --- m3-libs/m3core/src/Csupport/m3makefile.orig 2009-07-24 10:51:01.000000000 +0000 +++ m3-libs/m3core/src/Csupport/m3makefile @@ -11,6 +11,7 @@ readonly _CSupportPieces = { % "ALPHA_OSF" : [ "little-endian" ], % "AIX386" : [ "little-endian" ], "AMD64_DARWIN": [ "little-endian" ], + "AMD64_DRAGONFLY": [ "little-endian" ], "AMD64_FREEBSD": [ "little-endian" ], "AMD64_OPENBSD": [ "little-endian" ], "AMD64_NETBSD": [ "little-endian" ], --- m3-libs/m3core/src/float/m3makefile.orig 2009-07-24 10:51:24.000000000 +0000 +++ m3-libs/m3core/src/float/m3makefile @@ -13,6 +13,7 @@ _FloatPieces = { % "ALPHA_OSF" : _float_le, % "AIX386" : _float_le, "AMD64_DARWIN": _float_le, + "AMD64_DRAGONFLY": _float_le, "AMD64_FREEBSD": _float_le, "AMD64_OPENBSD": _float_le, "AMD64_NETBSD": _float_le, --- m3-libs/m3core/src/m3core.h.orig 2010-03-28 10:07:45.000000000 +0000 +++ m3-libs/m3core/src/m3core.h @@ -207,7 +207,8 @@ typedef INTEGER m3_uid_t; #define PTHREAD_TO_M3(x) ((m3_pthread_t)(size_t)(x)) #define PTHREAD_FROM_M3(x) ((pthread_t)(size_t)(x)) -#if defined(__APPLE__) || defined(__FreeBSD__) || defined(__NetBSD__) || defined(__OpenBSD__) +#if defined(__APPLE__) || defined(__FreeBSD__) || defined(__NetBSD__) \ + || defined(__OpenBSD__) || defined(__DragonFly__) #define HAS_STAT_FLAGS #endif --- m3-libs/m3core/src/platforms.quake.orig 2010-03-28 10:09:07.000000000 +0000 +++ m3-libs/m3core/src/platforms.quake @@ -1,5 +1,6 @@ _TARGET_OS = { "AMD64_DARWIN" : "DARWIN", + "AMD64_DRAGONFLY" : "DRAGONFLY", "AMD64_FREEBSD" : "FREEBSD", "AMD64_LINUX" : "LINUX", "AMD64_NETBSD" : "NETBSD", @@ -32,6 +33,7 @@ _TARGET_ENDIAN = { % This is not computed based on architecture % because some architectures vary, e.g. PPC_NT is little endian, PPC_DARWIN is big. "AMD64_DARWIN" : "LITTLE", + "AMD64_DRAGONFLY" : "LITTLE", "AMD64_FREEBSD" : "LITTLE", "AMD64_LINUX" : "LITTLE", "AMD64_NETBSD" : "LITTLE", --- m3-libs/m3core/src/runtime/POSIX/RTSignalC.c.orig 2010-04-16 12:20:56.000000000 +0000 +++ m3-libs/m3core/src/runtime/POSIX/RTSignalC.c @@ -167,7 +167,7 @@ static size_t GetPC(void* xcontext) #elif defined(__NetBSD__) _UC_MACHINE_PC(context) -#elif defined(__FreeBSD__) +#elif defined(__FreeBSD__) || defined(__DragonFly__) #if defined(__amd64) context->uc_mcontext.mc_rip #elif defined(__i386) --- m3-libs/m3core/src/runtime/common/Compiler.tmpl.orig 2009-07-13 09:09:35.000000000 +0000 +++ m3-libs/m3core/src/runtime/common/Compiler.tmpl @@ -21,7 +21,7 @@ readonly Compiler_i3 = [ " SPARC64_OPENBSD, PPC32_OPENBSD, MIPS64_OPENBSD,", " SPARC64_SOLARIS, I386_OPENBSD, AMD64_FREEBSD,", " PA32_HPUX, PA64_HPUX, ARM_DARWIN, I386_INTERIX,", - " AMD64_NETBSD, AMD64_OPENBSD };", + " AMD64_NETBSD, AMD64_OPENBSD, AMD64_DRAGONFLY };", "", "CONST", " ThisOS = OS." & OS_TYPE & ";", --- m3-libs/m3core/src/thread.quake.orig 2009-12-20 11:01:02.000000000 +0000 +++ m3-libs/m3core/src/thread.quake @@ -12,6 +12,7 @@ readonly NO_USER_THREADS = "AMD64_FREEBSD":1, "AMD64_OPENBSD":1, "AMD64_NETBSD":1, + "AMD64_DRAGONFLY":1, "AMD64_LINUX":1, "ARM_LINUX":1, "MIPS32_IRIX":1, --- m3-libs/m3core/src/thread/PTHREAD/ThreadFreeBSD.c.orig 2009-12-21 05:35:34.000000000 +0000 +++ m3-libs/m3core/src/thread/PTHREAD/ThreadFreeBSD.c @@ -2,7 +2,7 @@ /* All rights reserved. */ /* See the file COPYRIGHT-PURDUE for a full description. */ -#ifndef __FreeBSD__ +#if !defined(__FreeBSD__) && !defined(__DragonFly__) /* avoid empty file */ --- m3-libs/m3core/src/thread/PTHREAD/ThreadPThreadC.c.orig 2009-12-12 14:48:10.000000000 +0000 +++ m3-libs/m3core/src/thread/PTHREAD/ThreadPThreadC.c @@ -23,7 +23,7 @@ #endif #endif -#ifdef __FreeBSD__ +#if defined(__FreeBSD__) || defined(__DragonFly__) #define M3_DIRECT_SUSPEND #endif @@ -177,7 +177,7 @@ void ThreadPThread__sem_post(void) void ThreadPThread__sem_getvalue(void) { M3_DIRECT_SUSPEND_ASSERT_FALSE } void ThreadPThread__sigsuspend(void) { M3_DIRECT_SUSPEND_ASSERT_FALSE } -#ifdef __FreeBSD__ +#if defined(__FreeBSD__) || defined(__DragonFly__) int ThreadPThread__SuspendThread (m3_pthread_t mt) --- m3-libs/m3core/src/time/POSIX/m3makefile.orig 2009-07-24 07:26:15.000000000 +0000 +++ m3-libs/m3core/src/time/POSIX/m3makefile @@ -11,6 +11,7 @@ readonly _DateImpls = { % "ALPHA_OSF" : "DateBsd", % "AIX386" : "DateBsd", "AMD64_DARWIN": "DateBsd", + "AMD64_DRAGONFLY": "DateBsd", "AMD64_FREEBSD": "DateBsd", "AMD64_OPENBSD": "DateBsd", "AMD64_NETBSD": "DateBsd", @@ -63,6 +64,7 @@ readonly _DateImpls = { % "AIX" : "DateBsd", "CYGWIN" : "DatePosix", "DARWIN" : "DateBsd", + "DRAGONFLY" : "DataBsd", "FREEBSD" : "DateBsd", "HPUX" : "DatePosix", "INTERIX" : "DatePosix", --- m3-libs/m3core/src/unix/Common/UtimeC.c.orig 2009-12-11 11:18:34.000000000 +0000 +++ m3-libs/m3core/src/unix/Common/UtimeC.c @@ -56,7 +56,8 @@ struct timeval #endif } -#if defined(__OpenBSD__) || defined(__FreeBSD__) || defined(__NetBSD__) || defined(__APPLE__) +#if defined(__OpenBSD__) || defined(__FreeBSD__) || defined(__NetBSD__) \ + || defined(__APPLE__) || defined(__DragonFly__) #define M3BSD #endif --- m3-libs/m3core/src/unix/Common/Uucontext.c.orig 2009-06-29 19:20:40.000000000 +0000 +++ m3-libs/m3core/src/unix/Common/Uucontext.c @@ -29,6 +29,7 @@ see http://www.opengroup.org/onlinepubs/ || defined(__linux) \ || defined(__NetBSD__) \ || defined(__hpux) \ + || defined(__DragonFly__) \ || defined(__FreeBSD__) \ int Uucontext__getcontext(ucontext_t* context) --- m3-libs/m3core/src/unix/Common/m3makefile.orig 2009-07-19 00:28:17.000000000 +0000 +++ m3-libs/m3core/src/unix/Common/m3makefile @@ -81,7 +81,7 @@ if equal (TARGET, "MIPS64_OPENBSD") or e or equal (TARGET, "PA32_HPUX") or equal (TARGET, "PA64_HPUX") or equal (TARGET, "NT386") or equal (TARGET, "AMD64_FREEBSD") or equal (TARGET, "AMD64_OPENBSD") or equal (TARGET, "AMD64_NETBSD") - or equal (TARGET, "I386_INTERIX") + or equal (TARGET, "I386_INTERIX") or equal (TARGET, "AMD64_DRAGONFLY") Interface("Uucontext") --- m3-libs/m3core/src/unix/m3makefile.orig 2009-07-28 18:49:13.000000000 +0000 +++ m3-libs/m3core/src/unix/m3makefile @@ -6,6 +6,7 @@ readonly _UnixPieces = { % "AIX386" : [ "aix-ps2-1-2", "uin-common" ], % "ALPHA_OSF" : [ "osf-1.generic", "osf-1.ALPHA_OSF", "uin-common" ], "AMD64_DARWIN" : [ "darwin-common", "darwin-amd64", "uin-len" ], + "AMD64_DRAGONFLY" : [ "freebsd-common", "uin-len" ], "AMD64_FREEBSD" : [ "freebsd-common", "uin-len" ], "AMD64_LINUX" : [ "linux-common", "uin-common" ], "AMD64_NETBSD" : [ "netbsd-common", "uin-len" ], --- m3-sys/cminstall/src/config-no-install/AMD64_DRAGONFLY.orig 2014-01-16 23:14:23.000000000 +0000 +++ m3-sys/cminstall/src/config-no-install/AMD64_DRAGONFLY @@ -0,0 +1,9 @@ +readonly TARGET = "AMD64_DRAGONFLY" +readonly GNU_PLATFORM = "amd64-dragonfly" % "cpu-os" string for GNU + +m3back_flags = "-gstabs+ -m64 -fPIC -funwind-tables" +SYSTEM_CC = "gcc -gstabs+ -m64 -fPIC" +readonly SYSTEM_ASM = "as -64" + +include("AMD64.common") +include("DragonFly.common") --- m3-sys/cminstall/src/config-no-install/DragonFly.common.orig 2014-01-16 23:18:36.000000000 +0000 +++ m3-sys/cminstall/src/config-no-install/DragonFly.common @@ -0,0 +1,24 @@ +readonly TARGET_OS = "DRAGONFLY" + +GNU_MAKE = "gmake" + +include("Unix.common") + +SYSTEM_LIBS{"ODBC"} = [ "-Wl,-R/usr/local/lib", "-L/usr/local/lib", "-lodbc" ] +SYSTEM_LIBS{"POSTGRES95"} = [ "-Wl,-R/usr/local/lib", "-L/usr/local/lib", "-lpq" ] +SYSTEM_LIBS{"X11"} = ["-Wl,-R/usr/local/lib", "-L/usr/local/lib", "-lXaw", "-lXmu", "-lXext", "-lXt", "-lSM", "-lICE", "-lX11" ] + +DRAGONFLY_CC_APPEND = " -z origin" + & " -z now" + & " -Bsymbolic" + +DRAGONFLY_LD_APPEND = DRAGONFLY_CC_APPEND + & " -Wl,--fatal-warnings" + & " -Wl,--warn-common" + & " -Wl,-rpath,/usr/local/lib" + & " -Wl,-rpath,\\$ORIGIN/../lib " + +SYSTEM_LD = SYSTEM_CC & DRAGONFLY_LD_APPEND +SYSTEM_CC = SYSTEM_CC & DRAGONFLY_CC_APPEND + +include("gnuld.common") --- m3-sys/m3cc/src/boot.py.orig 2008-05-19 13:09:03.000000000 +0000 +++ m3-sys/m3cc/src/boot.py @@ -76,7 +76,7 @@ if SearchPath("gcc"): " + env) Run(gmake + "all-gmp all-mpfr all-libcpp all-libdecnumber \ - all-build-libiberty all-libiberty configure-gcc") + all-build-libiberty all-libiberty configure-libcpp configure-gcc") os.chdir("gcc") --- m3-sys/m3cc/src/m3makefile.orig 2010-05-14 07:00:52.000000000 +0000 +++ m3-sys/m3cc/src/m3makefile @@ -577,7 +577,7 @@ end if GCC42 m3cc_Run (["cd " & build_dir & " && " & M3CC_MAKE & " all-build-libiberty all-libiberty all-libdecnumber | tee -a " & Log]) else - m3cc_Run (["cd " & build_dir & " && " & M3CC_MAKE & " all-gmp all-mpfr all-build-libiberty all-libiberty all-libdecnumber | tee -a " & Log]) + m3cc_Run (["cd " & build_dir & " && " & M3CC_MAKE & " all-gmp all-mpfr all-build-libiberty all-libiberty all-libdecnumber configure-libcpp | tee -a " & Log]) end % --- m3-sys/m3cc/src/platforms.quake.orig 2009-07-29 17:11:08.000000000 +0000 +++ m3-sys/m3cc/src/platforms.quake @@ -13,6 +13,7 @@ readonly Platform_info = { "ALPHA_OSF" : "alpha-dec-osf1", "ALPHA_VMS" : "alpha-vms", "AMD64_DARWIN" : "i686-apple-darwin8", + "AMD64_DRAGONFLY" : "amd64-dragonfly", "AMD64_FREEBSD" : "amd64-freebsd7", "AMD64_LINUX" : "amd64-pc-linux-gnu", "AMD64_NETBSD" : "amd64-netbsd", --- m3-sys/m3gdb/src/platforms.quake.orig 2010-04-21 17:06:38.000000000 +0000 +++ m3-sys/m3gdb/src/platforms.quake @@ -13,6 +13,7 @@ readonly Platform_info = { "ALPHA_OSF" : "alpha-dec-osf1", "ALPHA_VMS" : "alpha-vms", "AMD64_DARWIN" : "amd64-apple-darwin8.7.1", + "AMD64_DRAGONFLY" : "amd64-dragonfly", "AMD64_FREEBSD" : "amd64-freebsd", "AMD64_LINUX" : "amd64-pc-linux-gnu", "AMD64_NETBSD" : "amd64-netbsd", --- m3-sys/m3middle/src/Target.i3.orig 2010-02-01 14:05:35.000000000 +0000 +++ m3-sys/m3middle/src/Target.i3 @@ -33,7 +33,7 @@ TYPE AMD64_DARWIN, AMD64_LINUX, SPARC32_LINUX, SPARC64_LINUX, SPARC64_OPENBSD, PPC32_OPENBSD, MIPS64_OPENBSD, SPARC64_SOLARIS, I386_OPENBSD, AMD64_FREEBSD, PA32_HPUX, PA64_HPUX, ARM_DARWIN, - I386_INTERIX, AMD64_NETBSD, AMD64_OPENBSD, Undefined + I386_INTERIX, AMD64_NETBSD, AMD64_OPENBSD, AMD64_DRAGONFLY, Undefined }; CONST @@ -89,7 +89,8 @@ CONST (* 48 *) "ARM_DARWIN", (* 49 *) "I386_INTERIX", (* 50 *) "AMD64_NETBSD", - (* 51 *) "AMD64_OPENBSD" + (* 51 *) "AMD64_OPENBSD", + (* 52 *) "AMD64_DRAGONFLY" }; CONST --- m3-sys/m3middle/src/Target.m3.orig 2010-02-01 15:06:05.000000000 +0000 +++ m3-sys/m3middle/src/Target.m3 @@ -307,6 +307,7 @@ PROCEDURE Init (system: TEXT; in_OS_name | Systems.AMD64_NETBSD, Systems.AMD64_OPENBSD, + Systems.AMD64_DRAGONFLY, Systems.AMD64_FREEBSD => Jumpbuf_size := 16_60 * Char.size; --- m3-tools/cvsup/suplib/src/FreeBSD/UProcTitle.c.orig 2009-04-12 10:31:23.000000000 +0000 +++ m3-tools/cvsup/suplib/src/FreeBSD/UProcTitle.c @@ -1,11 +1,11 @@ -#ifdef __FreeBSD__ +#if defined __FreeBSD__) || defined(__DragonFly__) #include #include #endif void m3_setproctitle(const char *title) { -#ifdef __FreeBSD__ +#if defined(__FreeBSD__) || defined(__DragonFly__) setproctitle("%s", title); #endif } --- scripts/boot-cm3-makefile-lib.tmpl.orig 2005-04-07 22:35:07.000000000 +0000 +++ scripts/boot-cm3-makefile-lib.tmpl @@ -20,9 +20,10 @@ FILES_C := $(wildcard *.c) FILES_S := $(wildcard *.s) BASES_IC := $(basename $(FILES_IC)) BASES_MC := $(basename $(FILES_MC)) +BASES_MS := $(basename $(FILES_MS)) BASES_C := $(basename $(FILES_C) $(FILES_S)) OBJ_IO := $(addsuffix .io, $(BASES_IC)) -OBJ_MO := $(addsuffix .mo, $(BASES_MC)) +OBJ_MO := $(addsuffix .mo, $(BASES_MS)) OBJ_O := $(addsuffix .o, $(BASES_C)) OBJECTS := $(OBJ_IO) $(OBJ_MO) $(OBJ_O) --- scripts/boot-cm3-makefile-prog.tmpl.orig 2003-07-10 22:16:05.000000000 +0000 +++ scripts/boot-cm3-makefile-prog.tmpl @@ -19,9 +19,10 @@ FILES_MS := $(wildcard *.ms) FILES_C := $(wildcard *.c) BASES_IC := $(basename $(FILES_IC)) BASES_MC := $(basename $(FILES_MC)) +BASES_MS := $(basename $(FILES_MS)) BASES_C := $(basename $(FILES_C)) OBJ_IO := $(addsuffix .io, $(BASES_IC)) -OBJ_MO := $(addsuffix .mo, $(BASES_MC)) +OBJ_MO := $(addsuffix .mo, $(BASES_MS)) OBJ_O := $(addsuffix .o, $(BASES_C)) OBJECTS := $(OBJ_IO) $(OBJ_MO) $(OBJ_O) LIBFILES := $(shell for f in $(LIBS); do if [ -f "$${f}" ] ; then echo $${f}; fi; done) --- scripts/pack-crossbuild.sh.orig 2010-02-25 14:02:18.000000000 +0000 +++ scripts/pack-crossbuild.sh @@ -23,7 +23,7 @@ if [ -z "$1" ] ; then fi CROSS_TARGET=$1 BUILDARGS='' -M3CONFIG=${ROOT}/m3-sys/cm3/src/config/${CROSS_TARGET} +M3CONFIG=${ROOT}/m3-sys/cminstall/src/config-no-install/${CROSS_TARGET} export M3CONFIG . "$ROOT/scripts/pkginfo.sh" . "$ROOT/scripts/pkgcmds.sh" @@ -38,25 +38,7 @@ P="${P} m3back" P="${P} m3front" P="${P} m3quake" P="${P} cm3" -P="${P} m3scanner" -P="${P} m3tools" -P="${P} m3cgcat" -P="${P} m3cggen" -[ "${M3GDB}" = yes ] && P="${P} m3gdb" P="${P} m3bundle" -P="${P} mklib" -P="${P} fix_nl" -P="${P} libdump" -P="${P} bitvector" -P="${P} digraph" -P="${P} parseparams" -P="${P} realgeometry" -P="${P} set" -P="${P} slisp" -P="${P} sortedtableextras" -P="${P} table-list" -P="${P} tempfiles" -[ "${HAVE_TCL}" = "yes" ] && P="${P} tcl" USAGE=" `basename $0` cross_target From jay.krell at cornell.edu Sun Jan 19 12:35:54 2014 From: jay.krell at cornell.edu (Jay K) Date: Sun, 19 Jan 2014 11:35:54 +0000 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: <52DBA2EF.4020101@marino.st> References: <52DBA2EF.4020101@marino.st> Message-ID: C:\dev2\cm3.2>findstr /s /i /p /c:"unrecognized target machine" *m3 m3-sys\cm3\src\Builder.m3: Msg.FatalError (NIL, "unrecognized target machin e: TARGET = ", s.target); Please don't bother with the last release. Please use the current CVS version. And please use the C backend. Did you edit m3-sys/m3middle/src/Target.i3 and possibly Target.m3? Please use this as a guide for files to consider changing: C:\dev2\cm3.2>findstr /m /s /i /p freebsd m3makefile m3-comm\udp\test\src\m3makefile m3-libs\libm3\src\os\POSIX\m3makefile m3-libs\m3core\src\float\m3makefile m3-libs\m3core\src\thread\m3makefile m3-libs\m3core\src\thread\PTHREAD\m3makefile m3-libs\m3core\src\unix\m3makefile m3-sys\m3cc\src\m3makefile m3-sys\m3gdb\src\m3makefile m3-tools\cvsup\suplib\src\FreeBSD\m3makefile m3-tools\cvsup\suplib\src\m3makefile m3-tools\cvsup\suptcp\src\POSIX\m3makefile C:\dev2\cm3.2>findstr /m /s /i /p freebsd *.c m3-libs\libm3\src\uid\POSIX\getMID.c m3-libs\libm3\src\uid\POSIX\MachineIDPosixC.c m3-libs\m3core\src\Csupport\Common\s_lroundl.c m3-libs\m3core\src\float\C99\FloatModeC.c m3-libs\m3core\src\runtime\common\RTProcessC.c m3-libs\m3core\src\runtime\POSIX\RTSignalC.c m3-libs\m3core\src\thread\POSIX\ThreadPosixC.c m3-libs\m3core\src\thread\PTHREAD\ThreadFreeBSD.c m3-libs\m3core\src\thread\PTHREAD\ThreadOpenBSD.c m3-libs\m3core\src\thread\PTHREAD\ThreadPThreadC.c m3-libs\m3core\src\unix\Common\Uin.c m3-libs\patternmatching\src\libglob\fnmatch.c esp. m3core and libm3. If we are sure of the name "AMD64_DRAGONFLY" I can go ahead and do it for you. But I realize there is much value in me NOT doing it. I thought maybe AMD64_DBSD or DFLYBSD, but DRAGFLY is ok. - Jay > Date: Sun, 19 Jan 2014 11:03:27 +0100 > From: adacore at marino.st > To: m3devel at elegosoft.com > Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) > > Hi all, > > I've been trying to port cm3 to DragonFly x86-64. > I'm able to build cm3cg on DragonFly, and use it to build libraries > (e.g. libm3core) that were targeted for AMD64_FREEBSD. > > However, when I set the HOST to AMD64_FREEBSD and TARGET to > AMD64_DRAGONFLY, I get the following error, "Fatal Error: unrecognized > target machine: TARGET = AMD64_DRAGONFLY" (context below) > > ========================================================= > > /work/lang/modula3/work/modula3-5.8.6/scripts/pkgmap.sh -c "/work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS " m3core > > ROOT = /work/lang/modula3/work/modula3-5.8.6 > > CM3_ROOT = /work/lang/modula3/work/modula3-5.8.6 > > M3GDB = yes > > M3OSTYPE = POSIX > > TARGET = AMD64_DRAGONFLY > > GCC_BACKEND = yes > > INSTALLROOT = /work/lang/modula3/work/stage/usr/local > > PKGSDB = /work/lang/modula3/work/modula3-5.8.6/scripts/PKGS > > GREP = egrep > > GMAKE = gmake > > TMPDIR = /var/tmp > > EXE = > > SL = / > > TAR = tar > > CM3ROOT = /work/lang/modula3/work/modula3-5.8.6 > > CM3VERSION = 5.8.6 > > CM3VERSIONNUM = 050806 > > CM3LASTCHANGED = 2010-04-11 > > FIND = /usr/bin/find > > EGREP = egrep > > === package m3-libs/m3core === > > +++ /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS +++ > > --- building in AMD64_DRAGONFLY --- > > > > ignoring ../src/m3overrides > > > > > > Fatal Error: unrecognized target machine: TARGET = AMD64_DRAGONFLY > > > > *** execution of /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS failed *** > > *** [assembler-code] Error code 1 > > ========================================================= > > I thought I added all the target definitions but apparently I've missed > something. I also realize the current repo looks a lot different, but I > wanted to use the last official release. > > I've attached a concatenated file of patches showing what I've changed. > Could somebody suggest what data is missing that causes the error above? > > Regards, > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Jan 19 12:40:17 2014 From: jay.krell at cornell.edu (Jay K) Date: Sun, 19 Jan 2014 11:40:17 +0000 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: References: <52DBA2EF.4020101@marino.st>, Message-ID: Your change looks about right. Did you first "upgrade" and existing install, with these changes, and then cross? That is what you must do. - Jay From: jay.krell at cornell.edu To: adacore at marino.st; m3devel at elegosoft.com Subject: RE: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) Date: Sun, 19 Jan 2014 11:35:54 +0000 C:\dev2\cm3.2>findstr /s /i /p /c:"unrecognized target machine" *m3 m3-sys\cm3\src\Builder.m3: Msg.FatalError (NIL, "unrecognized target machin e: TARGET = ", s.target); Please don't bother with the last release. Please use the current CVS version. And please use the C backend. Did you edit m3-sys/m3middle/src/Target.i3 and possibly Target.m3? Please use this as a guide for files to consider changing: C:\dev2\cm3.2>findstr /m /s /i /p freebsd m3makefile m3-comm\udp\test\src\m3makefile m3-libs\libm3\src\os\POSIX\m3makefile m3-libs\m3core\src\float\m3makefile m3-libs\m3core\src\thread\m3makefile m3-libs\m3core\src\thread\PTHREAD\m3makefile m3-libs\m3core\src\unix\m3makefile m3-sys\m3cc\src\m3makefile m3-sys\m3gdb\src\m3makefile m3-tools\cvsup\suplib\src\FreeBSD\m3makefile m3-tools\cvsup\suplib\src\m3makefile m3-tools\cvsup\suptcp\src\POSIX\m3makefile C:\dev2\cm3.2>findstr /m /s /i /p freebsd *.c m3-libs\libm3\src\uid\POSIX\getMID.c m3-libs\libm3\src\uid\POSIX\MachineIDPosixC.c m3-libs\m3core\src\Csupport\Common\s_lroundl.c m3-libs\m3core\src\float\C99\FloatModeC.c m3-libs\m3core\src\runtime\common\RTProcessC.c m3-libs\m3core\src\runtime\POSIX\RTSignalC.c m3-libs\m3core\src\thread\POSIX\ThreadPosixC.c m3-libs\m3core\src\thread\PTHREAD\ThreadFreeBSD.c m3-libs\m3core\src\thread\PTHREAD\ThreadOpenBSD.c m3-libs\m3core\src\thread\PTHREAD\ThreadPThreadC.c m3-libs\m3core\src\unix\Common\Uin.c m3-libs\patternmatching\src\libglob\fnmatch.c esp. m3core and libm3. If we are sure of the name "AMD64_DRAGONFLY" I can go ahead and do it for you. But I realize there is much value in me NOT doing it. I thought maybe AMD64_DBSD or DFLYBSD, but DRAGFLY is ok. - Jay > Date: Sun, 19 Jan 2014 11:03:27 +0100 > From: adacore at marino.st > To: m3devel at elegosoft.com > Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) > > Hi all, > > I've been trying to port cm3 to DragonFly x86-64. > I'm able to build cm3cg on DragonFly, and use it to build libraries > (e.g. libm3core) that were targeted for AMD64_FREEBSD. > > However, when I set the HOST to AMD64_FREEBSD and TARGET to > AMD64_DRAGONFLY, I get the following error, "Fatal Error: unrecognized > target machine: TARGET = AMD64_DRAGONFLY" (context below) > > ========================================================= > > /work/lang/modula3/work/modula3-5.8.6/scripts/pkgmap.sh -c "/work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS " m3core > > ROOT = /work/lang/modula3/work/modula3-5.8.6 > > CM3_ROOT = /work/lang/modula3/work/modula3-5.8.6 > > M3GDB = yes > > M3OSTYPE = POSIX > > TARGET = AMD64_DRAGONFLY > > GCC_BACKEND = yes > > INSTALLROOT = /work/lang/modula3/work/stage/usr/local > > PKGSDB = /work/lang/modula3/work/modula3-5.8.6/scripts/PKGS > > GREP = egrep > > GMAKE = gmake > > TMPDIR = /var/tmp > > EXE = > > SL = / > > TAR = tar > > CM3ROOT = /work/lang/modula3/work/modula3-5.8.6 > > CM3VERSION = 5.8.6 > > CM3VERSIONNUM = 050806 > > CM3LASTCHANGED = 2010-04-11 > > FIND = /usr/bin/find > > EGREP = egrep > > === package m3-libs/m3core === > > +++ /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS +++ > > --- building in AMD64_DRAGONFLY --- > > > > ignoring ../src/m3overrides > > > > > > Fatal Error: unrecognized target machine: TARGET = AMD64_DRAGONFLY > > > > *** execution of /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS failed *** > > *** [assembler-code] Error code 1 > > ========================================================= > > I thought I added all the target definitions but apparently I've missed > something. I also realize the current repo looks a lot different, but I > wanted to use the last official release. > > I've attached a concatenated file of patches showing what I've changed. > Could somebody suggest what data is missing that causes the error above? > > Regards, > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Sun Jan 19 12:53:53 2014 From: adacore at marino.st (John Marino) Date: Sun, 19 Jan 2014 12:53:53 +0100 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: References: <52DBA2EF.4020101@marino.st> Message-ID: <52DBBCD1.7020904@marino.st> On 1/19/2014 12:35, Jay K wrote: > C:\dev2\cm3.2>findstr /s /i /p /c:"unrecognized target machine" *m3 > m3-sys\cm3\src\Builder.m3: Msg.FatalError (NIL, "unrecognized > target machin > e: TARGET = ", s.target); > > > Please don't bother with the last release. > Please use the current CVS version. But the last release is what is in FreeBSD ports, the basis of both FreeBSD and DragonFly. And ports doesn't build from repos normally, although there is a facility to pull directly from github, but I don't like that method myself. If there was an official downloadable snapshot or release candidate, I could use that. Since 2010 was the last release, is another one coming soon? > And please use the C backend. > Did you edit m3-sys/m3middle/src/Target.i3 and possibly Target.m3? Yes, that was one of the many patches in the attachment I send in the previous mail. > Please use this as a guide for files to consider changing: > C:\dev2\cm3.2>findstr /m /s /i /p freebsd m3makefile > m3-comm\udp\test\src\m3makefile > m3-libs\libm3\src\os\POSIX\m3makefile > m3-libs\m3core\src\float\m3makefile > m3-libs\m3core\src\thread\m3makefile > m3-libs\m3core\src\thread\PTHREAD\m3makefile > m3-libs\m3core\src\unix\m3makefile > m3-sys\m3cc\src\m3makefile > m3-sys\m3gdb\src\m3makefile > m3-tools\cvsup\suplib\src\FreeBSD\m3makefile > m3-tools\cvsup\suplib\src\m3makefile > m3-tools\cvsup\suptcp\src\POSIX\m3makefile The only ones I haven't patched already are m3-libs\m3core\src\thread\PTHREAD\m3makefile and the cvsup ones. Those wouldn't cause the error. I also assume this is the top of the repo. > C:\dev2\cm3.2>findstr /m /s /i /p freebsd *.c > m3-libs\libm3\src\uid\POSIX\getMID.c > m3-libs\libm3\src\uid\POSIX\MachineIDPosixC.c > m3-libs\m3core\src\Csupport\Common\s_lroundl.c > m3-libs\m3core\src\float\C99\FloatModeC.c > m3-libs\m3core\src\runtime\common\RTProcessC.c > m3-libs\m3core\src\runtime\POSIX\RTSignalC.c > m3-libs\m3core\src\thread\POSIX\ThreadPosixC.c > m3-libs\m3core\src\thread\PTHREAD\ThreadFreeBSD.c > m3-libs\m3core\src\thread\PTHREAD\ThreadOpenBSD.c > m3-libs\m3core\src\thread\PTHREAD\ThreadPThreadC.c > m3-libs\m3core\src\unix\Common\Uin.c > m3-libs\patternmatching\src\libglob\fnmatch.c > > > esp. m3core and libm3. I haven't seen freebsd in these, so must be a difference between release and repo. > If we are sure of the name "AMD64_DRAGONFLY" I can go ahead and do it > for you. > But I realize there is much value in me NOT doing it. > I thought maybe AMD64_DBSD or DFLYBSD, but DRAGFLY is ok. Because AMD64_DRAGONFLY is too long? I'd prefer to keep it "AMD64_DRAGONFLY" because it matches the other targets and because of all the work I've already done. I'm really, really close to getting cm3 bootstrapped on DragonFly with 5.8.6. I believe if this "M3_BOOTSTRAP = true" version would compile, I could build cm3 and libraries on DragonFly with the cm3cg that's already built on it. Once I have a fully native compiler, then I can update to the latest repo, especially if some kind of snapshot is released. Thanks, John From jay.krell at cornell.edu Sun Jan 19 13:00:01 2014 From: jay.krell at cornell.edu (Jay K) Date: Sun, 19 Jan 2014 12:00:01 +0000 Subject: [M3devel] Cooperative suspend? In-Reply-To: <8BC1682A-3760-44D0-BA67-3A0F3EA10E27@m3w.org> References: <8BC1682A-3760-44D0-BA67-3A0F3EA10E27@m3w.org> Message-ID: Windows uses SuspendThread. I'll double check. Cooperative suspend would NOT involve sending a "signal" to threads, in the Posix sense. Nor would it use SuspendThread. The model for cooperative suspend has been described by Tony and goes something like this: There is a list of threads maintained by the m3 runtime. Not by enumerating OS threads. The thread structure contains in it perhaps a boolean or enum as to suspend requested, suspending, suspended, resuming, running, etc. When a thread wants to collect garbage, it either walks the thread list and sets all the booleans/enums to "suspend requested", or, more likely, it sets one global boolean. Now, there are three likely methods to set the global boolean. 1 - it is a global writable variable 2 - it is a global writable function pointer 3 - it is a page with protection Code frequently polls this global boolean, in one of the following fashions 1 - read and test the value of the variable 2 - call the function pointer 3 - read the page (just a byte or a word) ignoring the contents 4 - call a function to do any of the above, esp #1 #3 is probably fastest but my least favorite. It has lingering portability problems and is slowest when it comes time to actually suspend -- catch the exception and for purposes of debugging, you really don't want a design that raises exceptions like that. When does the polling occur? The polling is generated by the compiler. It is done at function entry and/or exit and at branches or at least backward branches or at least branches that are continuing loops or such or at the starts of loops or somesuch. I believe Java uses cooperative suspend and I think it uses the page protection mechanism. It isn't portable e.g. to systems without an MMU. Though of course you could abstract it out. I'm glossing over details, but you know all the code we have for signals and semaphores? It goes away. Our portability then would drive one more significant notch toward that of "hello world". In my mine, we have about two pieces to do to gain "hello world" level of portability - cooperative suspend, not easy - jmpbuf size removal, should be easy and some small matters - endianness, particularly in the libary - word size - Jay From: dragisha at m3w.org Date: Sat, 18 Jan 2014 09:24:11 +0100 To: m3devel at elegosoft.com Subject: [M3devel] Cooperative suspend? Jay, What exactly do you mean? Isn't it cooperative already? Thread stopping the world informs others about its intent, and they suspend themselves on a semaphore. Windows implementation of threading does forced suspend? dd --Dragi?a Duri?dragisha at m3w.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Sun Jan 19 13:09:37 2014 From: adacore at marino.st (John Marino) Date: Sun, 19 Jan 2014 13:09:37 +0100 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: References: <52DBA2EF.4020101@marino.st>, Message-ID: <52DBC081.1070401@marino.st> On 1/19/2014 12:40, Jay K wrote: > Your change looks about right. > > Did you first "upgrade" and existing install, with these changes, and > then cross? > That is what you must do. It's probably easier that I show you. I've attached "Makefile.local" (I added the .txt extension just now to help out some mail clients) which is my recipe. So I run three targets in a freshly patched workarea: 1) make cross-compiler 2) make assembler-code (called from cross-archive) 3) make cross-archive In the cross-compiler target, the FreeBSD bootstrap compiler builds m3middle, m3linker, m3front, mequake, and CM3 for the cross compiler. The cross-compiler then builds libm3core and libm3 with FreeBSD cm3cg Everything is cleaned, then the cross compiler builds everything again (including libm3core and libm3) along with m3objfile, sysutils, and patternmatching. Then the cm3.cfg is changed to specify HOST=freebsd, TARGET=dragonfly, cm3cg is cm3cg-AMD64_DRAGONFLY. I would not be surprised is there is a logic flaw or two in "make cross-compiler". The "how to port CM3" tutorial was slightly out of date and some of the instructions were a bit ambiguous so this was a best guess. Regards, John -------------- next part -------------- # This makefile fragment is automatically included by bsd.port.mk # It is separate because it contains maintenance targets not used in # normal package building, but the recipes are too value to lose. So # we keep them with the port, but out of the way of regular users. # # $FreeBSD$ ## # # The "create-bootstrap" target will assemble a new bootstrap based on the # contents of the stage directory, which means it has to be run after the # "stage" target. # ## NEWBOOTDIR= ${WRKDIR}/new-bootstrap/bootstrap NEWBOOTNAME= m3-bootstrap.${MARCH}.${OPSYS:U}.${OSREL:S/.//}.tar.bz2 BSCONTENTS= bin/cm3 bin/cm3cg bin/m3bundle bin/mklib etc/modula3 \ lib/libm3core.* lib/libm3.* lib/libsysutils.* \ lib/libpatternmatching.* pkg/m3core pkg/libm3 pkg/sysutils \ pkg/patternmatching pkg/m3middle pkg/m3objfile pkg/m3linker \ pkg/m3back pkg/m3front pkg/m3quake pkg/cm3 pkg/mklib create-bootstrap: @${RM} -rf ${NEWBOOTDIR} @${MKDIR} ${NEWBOOTDIR}/bin ${NEWBOOTDIR}/lib \ ${NEWBOOTDIR}/pkg ${NEWBOOTDIR}/etc .for X in ${BSCONTENTS} @${CP} -a ${STAGEDIR}${PREFIX}/${X} ${NEWBOOTDIR}/${X:H}/ .endfor ${ECHO} "INSTALL_ROOT = path() & \"/..\"" \ > ${NEWBOOTDIR}/bin/cm3.cfg ${ECHO} "include(path() & \"/../etc/modula3/${M3TARGET}\")" \ >> ${NEWBOOTDIR}/bin/cm3.cfg @${FIND} ${NEWBOOTDIR} -type f -perm +111 | ${XARGS} ${STRIP_CMD} (cd ${NEWBOOTDIR}/.. ; tar -cyf ${NEWBOOTNAME} bootstrap) ## # # The "cross-compiler" target will create a cross-compiler in the # stage directory based on the value of ${CROSSTARG}. # Execute it after the "patch" target # # The "cross-archive" will archive assembler code meant for linking # on the native plaform. It is run after "cross-compiler" # # The "native-link" target will bootstrap cm3 on a new platform (which # obviously supports ports) using the archived files generated above. # It could be done remotely via SSH, but this one target assumes files # are locally available at ${WRKSRC}/${CROSSTARG} # ## CROSSTARG?= AMD64_DRAGONFLY CORELIBS= m3-libs/m3core m3-libs/libm3 CORESYS= m3-sys/m3middle m3-sys/m3objfile m3-sys/m3linker \ m3-sys/m3back m3-sys/m3front m3-sys/m3quake m3-sys/cm3 \ m3-tools/m3bundle COREHELPERS= m3-libs/sysutils m3-libs/patternmatching COREFILES= ${CORELIBS} ${CORESYS} cross-compiler: @(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BOOTCM3} SHIP=${TRUE} \ ${SH} scripts/build-cross-backend.sh ${CROSSTARG}) @(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BOOTCM3} SHIP=${TRUE} \ ${SH} scripts/do-pkg.sh onlybuild \ m3middle m3linker m3front m3quake cm3) @${FIND} ${WRKSRC} -name \.M3SHIP -print | ${XARGS} ${SED} -i -e \ 's|/bootstrap/|/stage${PREFIX}/|g' @(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BOOTCM3} \ ${SH} scripts/do-pkg.sh ship \ m3middle m3linker m3front m3quake cm3) (cd ${WRKSRC}/m3-sys/cminstall/src/config-no-install && \ ${COPYTREE_SHARE} . ${STAGEDIR}${PREFIX}/etc/modula3) ${MKDIR} ${STAGEDIR}${PREFIX}/bin ${INSTALL_PROGRAM} ${WRKSRC}/m3-sys/cm3/${M3TARGET}/cm3 \ ${STAGEDIR}${PREFIX}/bin ${ECHO} "INSTALL_ROOT = \"${STAGEDIR}${PREFIX}\"" > \ ${STAGEDIR}${PREFIX}/bin/cm3.cfg ${ECHO} "include(\"${STAGEDIR}${PREFIX}/etc/modula3/${M3TARGET}\")" >> \ ${STAGEDIR}${PREFIX}/bin/cm3.cfg ${INSTALL_PROGRAM} ${WRKDIR}/bootstrap/bin/cm3cg \ ${STAGEDIR}${PREFIX}/bin/ ${INSTALL_PROGRAM} \ ${WRKSRC}/m3-sys/m3cc/${M3TARGET}-${CROSSTARG}/cm3cg \ ${STAGEDIR}${PREFIX}/bin/cm3cg-${CROSSTARG} @(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BOOTCM3} \ ${SH} scripts/do-pkg.sh cleanglobal \ m3middle m3linker m3front m3quake cm3) .for component in ${CORELIBS} @(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BUILTCM3} \ ${SH} scripts/do-pkg.sh buildship ${component:T}) .endfor @(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BUILTCM3} \ ${SH} scripts/do-pkg.sh cleanglobal ${CORELIBS:T}) .for component in ${CORELIBS} ${COREHELPERS} ${CORESYS} @(cd ${WRKSRC}; \ ${SETENV} ${M3MAKE_ENV} ${BUILTCM3} \ ${SH} scripts/do-pkg.sh buildship ${component:T}; \ ${SETENV} ${M3MAKE_ENV} ${BUILTCM3} \ ${SH} scripts/do-pkg.sh cleanglobal ${component:T} \ ) .endfor ${ECHO} "INSTALL_ROOT = \"${STAGEDIR}${PREFIX}\"" > \ ${STAGEDIR}${PREFIX}/bin/cm3.cfg ${ECHO} "m3back = INSTALL_ROOT & \"/bin/cm3cg-${CROSSTARG}\"" >> \ ${STAGEDIR}${PREFIX}/bin/cm3.cfg ${ECHO} "BUILD_DIR = \"${CROSSTARG}\"" >> \ ${STAGEDIR}${PREFIX}/bin/cm3.cfg ${ECHO} "HOST = \"${M3TARGET}\"" >> ${STAGEDIR}${PREFIX}/bin/cm3.cfg ${ECHO} "include(\"${STAGEDIR}${PREFIX}/etc/modula3/${CROSSTARG}\")" >> \ ${STAGEDIR}${PREFIX}/bin/cm3.cfg ${ECHO} "M3_BOOTSTRAP = TRUE" >> ${STAGEDIR}${PREFIX}/bin/cm3.cfg @${ECHO} "=====================================" @${ECHO} "===== M3 cross-compiler ready =====" @${ECHO} "=====================================" @${ECHO} assembler-code: .for component in ${COREFILES} (cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} SHIP=${TRUE} TARGET=${CROSSTARG} \ ${BUILTCM3} ${SH} scripts/do-pkg.sh onlybuild ${component:T} \ ) .endfor .for component in m3-libs/libm3 m3-sys/m3middle echo '#include "m3core.h"' > \ ${WRKSRC}/${component}/${CROSSTARG}/m3unix.h .endfor ${CP} ${WRKSRC}/m3-libs/m3core/src/m3core.h \ ${WRKSRC}/m3-libs/libm3/${CROSSTARG}/ (cd ${WRKSRC}/m3-sys/m3middle; ${CP} src/POSIX/m3core.h ${CROSSTARG}/) cross-archive: assembler-code (cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} M3GDB=no ${BUILTCM3} \ ${SH} scripts/pack-crossbuild.sh ${CROSSTARG}) native-cm3cg: # This target requires python to be installed @(cd ${WRKSRC}/m3-sys/m3cc; src/boot.py) @${MKDIR} ${STAGEDIR}${PREFIX}/bin ${INSTALL_PROGRAM} ${WRKSRC}/m3-sys/m3cc/obj.1/gcc/m3cgc1 \ ${STAGEDIR}${PREFIX}/bin/cm3cg native-link: native-cm3cg .for component in ${COREFILES} @(cd ${WRKSRC}/${CROSSTARG}; ${TAR} -xzf ${component:T}.tgz) .endfor @(cd ${WRKSRC}/scripts; \ ${SETENV} PATH=${PATH}:${STAGEDIR}${PREFIX}/bin \ ./boot-cm3-build-on-target.sh ${CROSSTARG}) ${INSTALL_PROGRAM} ${WRKSRC}/${CROSSTARG}/m3-sys/cm3/${CROSSTARG}/cm3 \ ${STAGEDIR}${PREFIX}/bin ${INSTALL_PROGRAM} ${WRKSRC}/${CROSSTARG}/m3-tools/m3bundle/${CROSSTARG}/m3bundle \ ${STAGEDIR}${PREFIX}/bin From jay.krell at cornell.edu Sun Jan 19 13:17:24 2014 From: jay.krell at cornell.edu (Jay K) Date: Sun, 19 Jan 2014 12:17:24 +0000 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: <52DBC081.1070401@marino.st> References: <52DBA2EF.4020101@marino.st>, , <52DBC081.1070401@marino.st> Message-ID: Hey..I like that that does capture some of My Way. Here is my independent off the cuff write up though: I know the ports system is nice. I know we really need to be in it for many operating systems. I suspect your problem is that you didn't "upgrade" to apply you changes at the right time. Do any of these work for you: https://modula3.elegosoft.com/cm3/snaps/snapshot-index.html Anyway, please try this: Ignore DragonFly at first. Go to a system with a working cm3. Such as FreeBSD. Linux. Almost anything. Such as from ports. Make a backup, e.g. of /usr/local/cm3. My instructions are unfortunately destructive of the existing install. Checkout the full current CVS repository. In that repository, run scripts/python/upgrade.py. If that doesn't work, stop, and we'll help. If that does work, apply your diffs. And upgrade.py again. You can say "upgrade.py skipgcc" here. Then boot1.py c amd64_dragonflybsd Make sure you say "c" and the target "amd64_dragonflybsd", anywhere on the command line. Ok, "c" is actually up to you. Since nobody messes with ABI much, gcc/amd64 works for the same for Linux, FreeBSD, OpenBSD, NetBSD, and Dragonfly. If this works, it will give you, roughly amd64_dragonflybsd*.tar.gz Copy that to your Dragonfly system, extract it, cd into it, make. If that works, you have a native cm3. Run it. It should say "can't find cm3.cfg". Now on Dragonfly, mkdir -p /usr/local/cm3/bin cp cm3 /usr/local/cm3/bin export PATH=/usr/local/cm3/bin:$PATH full CVS checkout cd scripts/python ; ./boot2.sh You will need to have made small changes to pylib.py for this to work. (And your config file should say to use the C backend, like Darwin and ARM_LINUX do.) - Jay > Date: Sun, 19 Jan 2014 13:09:37 +0100 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) > > On 1/19/2014 12:40, Jay K wrote: > > Your change looks about right. > > > > Did you first "upgrade" and existing install, with these changes, and > > then cross? > > That is what you must do. > > > It's probably easier that I show you. > I've attached "Makefile.local" (I added the .txt extension just now to > help out some mail clients) which is my recipe. > > So I run three targets in a freshly patched workarea: > 1) make cross-compiler > 2) make assembler-code (called from cross-archive) > 3) make cross-archive > > In the cross-compiler target, the FreeBSD bootstrap compiler builds > m3middle, m3linker, m3front, mequake, and CM3 for the cross compiler. > > The cross-compiler then builds libm3core and libm3 with FreeBSD cm3cg > Everything is cleaned, then the cross compiler builds everything again > (including libm3core and libm3) along with m3objfile, sysutils, and > patternmatching. > > Then the cm3.cfg is changed to specify HOST=freebsd, TARGET=dragonfly, > cm3cg is cm3cg-AMD64_DRAGONFLY. > > I would not be surprised is there is a logic flaw or two in "make > cross-compiler". The "how to port CM3" tutorial was slightly out of > date and some of the instructions were a bit ambiguous so this was a > best guess. > > Regards, > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Sun Jan 19 13:38:32 2014 From: adacore at marino.st (John Marino) Date: Sun, 19 Jan 2014 13:38:32 +0100 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: References: <52DBA2EF.4020101@marino.st>, , <52DBC081.1070401@marino.st> Message-ID: <52DBC748.5090308@marino.st> On 1/19/2014 13:17, Jay K wrote: > Hey..I like that that does capture some of My Way. > Here is my independent off the cuff write up though: > I know the ports system is nice. > I know we really need to be in it for many operating systems. > I suspect your problem is that you didn't "upgrade" to apply you > changes at the right time. I must be misunderstanding what you mean by "upgrade". It's the cross-compiler that's complaining, and it was built by the latest patches. > Do any of these work for you: > https://modula3.elegosoft.com/cm3/snaps/snapshot-index.html I'm not clear to which set you are referring. There's no need for a binary set, modula3 now builds on FreeBSD 9 and FreeBSD 10 (http://www.freshports.org/lang/modula3), which is also superior to the release binaries because they link gmp and mpfr statically while a bug in the cm3 sources caused those libraries to link dynamically (which is why release binaries don't run on current FreeBSD) Then there are the source distributions, but they are from 2012 so they aren't exactly fresh either. > Anyway, please try this: > Ignore DragonFly at first. > Go to a system with a working cm3. Such as FreeBSD. Linux. Almost > anything. > Such as from ports. > Make a backup, e.g. of /usr/local/cm3. > My instructions are unfortunately destructive of the existing install. > Checkout the full current CVS repository. > In that repository, run scripts/python/upgrade.py. > If that doesn't work, stop, and we'll help. Is the goal just to get FreeBSD to have a binary with the latest sources? If so, I'd just modify the port to do that now that we have working binary bootstraps. The main reason I'm reluctant is that it took over a week to make the cm3 port, I've never had so much difficulty before. (I've also spent another week on the DragonFly port). So I'm "resisting" because of how much work it already took. That's why I was trying to port DragonFly, THEN update to latest rather than update to latest and then port DragonFly. (in other words, going forward seems a short path than backtracking) > If that does work, apply your diffs. And upgrade.py again. You can say > "upgrade.py skipgcc" here. > Then boot1.py c amd64_dragonflybsd > Make sure you say "c" and the target "amd64_dragonflybsd", anywhere > on the command line. > Ok, "c" is actually up to you. Since nobody messes with ABI much, > gcc/amd64 works for the same > for Linux, FreeBSD, OpenBSD, NetBSD, and Dragonfly. I assume you mean after the system understands this target. There will be a lot of files with __FreeBSD__ that need to be changed to __FreeBSD__ || __DragonFly__ as well. In any case, I'll have to regenerate a bunch of patches from my current work. > If this works, it will give you, roughly amd64_dragonflybsd*.tar.gz > Copy that to your Dragonfly system, extract it, cd into it, make. > If that works, you have a native cm3. > Run it. It should say "can't find cm3.cfg". > Now on Dragonfly, mkdir -p /usr/local/cm3/bin > cp cm3 /usr/local/cm3/bin > export PATH=/usr/local/cm3/bin:$PATH > full CVS checkout > cd scripts/python ; ./boot2.sh > > You will need to have made small changes to pylib.py for this to work. > (And your config file should say to use the C backend, like Darwin and > ARM_LINUX do.) Well, I'm thrilled if I don't have to patch gcc again, that takes a long time. Shouldn't FreeBSD use the C backend too? John From jay.krell at cornell.edu Sun Jan 19 13:32:44 2014 From: jay.krell at cornell.edu (Jay K) Date: Sun, 19 Jan 2014 12:32:44 +0000 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: References: <52DBA2EF.4020101@marino.st>, , , , , <52DBC081.1070401@marino.st>, Message-ID: Clarification: upgrade.sh (note the .sh) rebuilds everything I wrote upgrade.py based on it, but missed the "everything" part. upgrade.py only upgrades "the compiler" (and m3core, libm3..) A better practise although inefficient is: upgrade.py && ./do-cm3-all.py realclean skipgcc && ./do-cm3-all.py buildship I didn't suggest that to you here in order to get further in the pipeline faster. However upgrade.py w/o other rebuild could leave m3core.so too new for many programs. But for purposes of building/cross-building just the compiler, my instructions are good. John, I see you added cm3 to the FreeBSD ports and you clearly know what you are doing. Nevertheless, I am a lazy snob, don't quite see what you are doing wrong, and am suggesting "just different", not clearly better. But, to my credit, this "just different" is what I use all the time. I have used it to cross to several systems, w/ and w/o cm3cg (w/ and w/o C backend). It isn't fast, and it doesn't always have the best incrementality, but it does usually work. There is redundancy. pylib.py essentially duplicates parts of the config files. You misunderstood what I meant by "C backend" and from the ports history you will somewhat appreciate it. The C backend, instead of having a lot to do with gcc, is a backend that generates C, and then invokes a C compiler. The problem you had with -gstabs for example, goes away. We'll use whatever debugging format regular C/C++ use. There is a little hope of taking the one C output and compiling it for different targets -- at least FreeBSD 8,9,10. Currently the C is very word size dependent, endian dependent, and somewhat otherwise target-dependent. I'd like to fix these aspects but it isn't easy by far, will take possibly controversial changes in the frontend and interface to backend. - Jay From: jay.krell at cornell.edu To: adacore at marino.st Date: Sun, 19 Jan 2014 12:17:24 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) Hey..I like that that does capture some of My Way. Here is my independent off the cuff write up though: I know the ports system is nice. I know we really need to be in it for many operating systems. I suspect your problem is that you didn't "upgrade" to apply you changes at the right time. Do any of these work for you: https://modula3.elegosoft.com/cm3/snaps/snapshot-index.html Anyway, please try this: Ignore DragonFly at first. Go to a system with a working cm3. Such as FreeBSD. Linux. Almost anything. Such as from ports. Make a backup, e.g. of /usr/local/cm3. My instructions are unfortunately destructive of the existing install. Checkout the full current CVS repository. In that repository, run scripts/python/upgrade.py. If that doesn't work, stop, and we'll help. If that does work, apply your diffs. And upgrade.py again. You can say "upgrade.py skipgcc" here. Then boot1.py c amd64_dragonflybsd Make sure you say "c" and the target "amd64_dragonflybsd", anywhere on the command line. Ok, "c" is actually up to you. Since nobody messes with ABI much, gcc/amd64 works for the same for Linux, FreeBSD, OpenBSD, NetBSD, and Dragonfly. If this works, it will give you, roughly amd64_dragonflybsd*.tar.gz Copy that to your Dragonfly system, extract it, cd into it, make. If that works, you have a native cm3. Run it. It should say "can't find cm3.cfg". Now on Dragonfly, mkdir -p /usr/local/cm3/bin cp cm3 /usr/local/cm3/bin export PATH=/usr/local/cm3/bin:$PATH full CVS checkout cd scripts/python ; ./boot2.sh You will need to have made small changes to pylib.py for this to work. (And your config file should say to use the C backend, like Darwin and ARM_LINUX do.) - Jay > Date: Sun, 19 Jan 2014 13:09:37 +0100 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) > > On 1/19/2014 12:40, Jay K wrote: > > Your change looks about right. > > > > Did you first "upgrade" and existing install, with these changes, and > > then cross? > > That is what you must do. > > > It's probably easier that I show you. > I've attached "Makefile.local" (I added the .txt extension just now to > help out some mail clients) which is my recipe. > > So I run three targets in a freshly patched workarea: > 1) make cross-compiler > 2) make assembler-code (called from cross-archive) > 3) make cross-archive > > In the cross-compiler target, the FreeBSD bootstrap compiler builds > m3middle, m3linker, m3front, mequake, and CM3 for the cross compiler. > > The cross-compiler then builds libm3core and libm3 with FreeBSD cm3cg > Everything is cleaned, then the cross compiler builds everything again > (including libm3core and libm3) along with m3objfile, sysutils, and > patternmatching. > > Then the cm3.cfg is changed to specify HOST=freebsd, TARGET=dragonfly, > cm3cg is cm3cg-AMD64_DRAGONFLY. > > I would not be surprised is there is a logic flaw or two in "make > cross-compiler". The "how to port CM3" tutorial was slightly out of > date and some of the instructions were a bit ambiguous so this was a > best guess. > > Regards, > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Jan 19 13:50:27 2014 From: jay.krell at cornell.edu (Jay K) Date: Sun, 19 Jan 2014 12:50:27 +0000 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: <52DBC748.5090308@marino.st> References: <52DBA2EF.4020101@marino.st>, , <52DBC081.1070401@marino.st> , <52DBC748.5090308@marino.st> Message-ID: I understand your reluctance. > Is the goal just to get FreeBSD to have a binary with the latest > sources? Yes, and to make sure the build process works for you. > Shouldn't FreeBSD use the C backend too? Every target should. I haven't convinced everyone yet. There is less resistance for new targets, like AMD64_NT and ARM_LINUX, which are working, using the C backend. There are downsides to the C backend: - it is slower to compiler - debugging is degraded on systems that have m3gdb (debugging is much better on systems w/o m3gdb) And there are advantages not yet implemented, in particular, efficient exception handling by generating C++. > > gcc/amd64 works for the same > > for Linux, FreeBSD, OpenBSD, NetBSD, and Dragonfly. > > I assume you mean after the system understands this target. There will > be a lot of files with __FreeBSD__ that need to be changed to > __FreeBSD__ || __DragonFly__ as well. In any case, I'll have to > regenerate a bunch of patches from my current work. Your patches should apply fairly cleanly to any new checkout. I am willing to through them and apply them to CVS if you want -- depends on if you are ok with my stealing your credit, vs. you want to commit them yourselves. I already skimmed them and they all look easy. DragonflyBSD is basically FreeBSD by another name. Yes, a ton of work has been done on the kernel, file systems, maybe ports. But the ABI is almost the same -- heck, for the most part, Linux==NetBSD==FreeBSD==OpenBSD. Sure, they might be implemented differently, but they are highly source compatible and every highly object code compatible. > I must be misunderstanding what you mean by "upgrade". It's the > cross-compiler that's complaining, and it was built by the latest patches. Agreed, we are both missing something simple. Usually this is the error you get when you don't upgrade Target.i3/Target.m3. If you want to press on with the current approach and ignore my advise, it should be possible to step through all this. To start, break on Target__Init and see if it returns true or false. > statically while a bug in the cm3 sources caused those libraries to link > dynamically (which is why release binaries don't run on current FreeBSD) Ugh. I saw something about that in your diffs. Definitely we intend to link statically. I actually went to the trouble of removing use of gmp/mpfr/mpc in current source. They are used for actually very little and aren't worth it. - Jay > Date: Sun, 19 Jan 2014 13:38:32 +0100 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) > > On 1/19/2014 13:17, Jay K wrote: > > Hey..I like that that does capture some of My Way. > > Here is my independent off the cuff write up though: > > I know the ports system is nice. > > I know we really need to be in it for many operating systems. > > I suspect your problem is that you didn't "upgrade" to apply you > > changes at the right time. > > I must be misunderstanding what you mean by "upgrade". It's the > cross-compiler that's complaining, and it was built by the latest patches. > > > > Do any of these work for you: > > https://modula3.elegosoft.com/cm3/snaps/snapshot-index.html > > I'm not clear to which set you are referring. > There's no need for a binary set, modula3 now builds on FreeBSD 9 and > FreeBSD 10 (http://www.freshports.org/lang/modula3), which is also > superior to the release binaries because they link gmp and mpfr > statically while a bug in the cm3 sources caused those libraries to link > dynamically (which is why release binaries don't run on current FreeBSD) > > Then there are the source distributions, but they are from 2012 so they > aren't exactly fresh either. > > > > Anyway, please try this: > > Ignore DragonFly at first. > > Go to a system with a working cm3. Such as FreeBSD. Linux. Almost > > anything. > > Such as from ports. > > Make a backup, e.g. of /usr/local/cm3. > > My instructions are unfortunately destructive of the existing install. > > Checkout the full current CVS repository. > > In that repository, run scripts/python/upgrade.py. > > If that doesn't work, stop, and we'll help. > > Is the goal just to get FreeBSD to have a binary with the latest > sources? If so, I'd just modify the port to do that now that we have > working binary bootstraps. The main reason I'm reluctant is that it > took over a week to make the cm3 port, I've never had so much difficulty > before. (I've also spent another week on the DragonFly port). So I'm > "resisting" because of how much work it already took. That's why I was > trying to port DragonFly, THEN update to latest rather than update to > latest and then port DragonFly. (in other words, going forward seems a > short path than backtracking) > > > > > If that does work, apply your diffs. And upgrade.py again. You can say > > "upgrade.py skipgcc" here. > > Then boot1.py c amd64_dragonflybsd > > Make sure you say "c" and the target "amd64_dragonflybsd", anywhere > > on the command line. > > Ok, "c" is actually up to you. Since nobody messes with ABI much, > > gcc/amd64 works for the same > > for Linux, FreeBSD, OpenBSD, NetBSD, and Dragonfly. > > I assume you mean after the system understands this target. There will > be a lot of files with __FreeBSD__ that need to be changed to > __FreeBSD__ || __DragonFly__ as well. In any case, I'll have to > regenerate a bunch of patches from my current work. > > > > If this works, it will give you, roughly amd64_dragonflybsd*.tar.gz > > Copy that to your Dragonfly system, extract it, cd into it, make. > > If that works, you have a native cm3. > > Run it. It should say "can't find cm3.cfg". > > Now on Dragonfly, mkdir -p /usr/local/cm3/bin > > cp cm3 /usr/local/cm3/bin > > export PATH=/usr/local/cm3/bin:$PATH > > full CVS checkout > > cd scripts/python ; ./boot2.sh > > > > You will need to have made small changes to pylib.py for this to work. > > (And your config file should say to use the C backend, like Darwin and > > ARM_LINUX do.) > > > Well, I'm thrilled if I don't have to patch gcc again, that takes a long > time. Shouldn't FreeBSD use the C backend too? > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Sun Jan 19 14:06:02 2014 From: adacore at marino.st (John Marino) Date: Sun, 19 Jan 2014 14:06:02 +0100 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: References: <52DBA2EF.4020101@marino.st>, , <52DBC081.1070401@marino.st> , <52DBC748.5090308@marino.st> Message-ID: <52DBCDBA.8030809@marino.st> On 1/19/2014 13:50, Jay K wrote: > And there are advantages not yet implemented, in particular, efficient > exception handling by generating C++. Like FreeBSD (and unlike NetBSD and OpenBSD), DragonFly has implemented dl_iterate_phdr which gcc uses for efficient zero-cost exception handling. That might come in handy here (I implemented it in DragonFly's real-time linker, mainly for Ada/GNAT but C++ benefited.) > Your patches should apply fairly cleanly to any new checkout. > I am willing to through them and apply them to CVS if you want -- > depends on if you are ok with my stealing your credit, vs. you want to > commit them yourselves. I would love it if you did this - I'm happy with a mention in the commit message. I just attached a tarball of all the patches I'm using right now. Some of these changes are specifically for the cross compiler (I'm think of the /scripts changes mostly) but most are valid assuming they still apply to the head of the repo. Please take as much as you want with my gratitude. > I already skimmed them and they all look easy. > DragonflyBSD is basically FreeBSD by another name. That's not an accident. As a DragonFly developer that mostly works on userland, I've been converging FreeBSD and DragonFly again. They drifted apart after 10 years, but I thought it better to keep the userlands as similar as possible for 3rd party applications. > Yes, a ton of work has been done on the kernel, file systems, maybe ports. > But the ABI is almost the same -- heck, for the most part, > Linux==NetBSD==FreeBSD==OpenBSD. > Sure, they might be implemented differently, but they are highly source > compatible and every highly object code compatible. >> I must be misunderstanding what you mean by "upgrade". It's the >> cross-compiler that's complaining, and it was built by the latest patches. > Agreed, we are both missing something simple. > Usually this is the error you get when you don't upgrade > Target.i3/Target.m3. > If you want to press on with the current approach and ignore my advise, I'm not ignoring it; it's more like I'm not quite ready to give up since I'm so close. I think I could actually get these libs and cm3 to build on DragonFly if I used AMD64_FREEBSD as the target with the cm3cg-AMD_DRAGONFLY generator. It was building before. I just figured I'd then get stuck on the DragonFly side trying to compile cm3 with itself, theoretically I should get the same error there. > Ugh. I saw something about that in your diffs. Definitely we intend to > link statically. > I actually went to the trouble of removing use of gmp/mpfr/mpc in > current source. > They are used for actually very little and aren't worth it. Even better! Regards, John -------------- next part -------------- A non-text attachment was scrubbed... Name: files.tar Type: application/x-tar Size: 13062 bytes Desc: not available URL: From dragisha at m3w.org Sun Jan 19 15:28:18 2014 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 19 Jan 2014 15:28:18 +0100 Subject: [M3devel] Cooperative suspend? In-Reply-To: References: <8BC1682A-3760-44D0-BA67-3A0F3EA10E27@m3w.org> Message-ID: Why would we do this, and not signalling, as in POSIX implementation? Why try to make something synchronous when we have working asynchronous implementation? Why would it be better than sync one? On 19 Jan 2014, at 13:00, Jay K wrote: > There is a list of threads maintained by the m3 runtime. > Not by enumerating OS threads. > The thread structure contains in it perhaps a boolean or enum as to suspend requested, suspending, suspended, resuming, running, etc. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Sun Jan 19 21:05:48 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sun, 19 Jan 2014 12:05:48 -0800 Subject: [M3devel] Win32 SuspendThread? In-Reply-To: <9EC3F463-F248-4C79-AF3F-130D2252B8B0@gmail.com> References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL> <9EC3F463-F248-4C79-AF3F-130D2252B8B0@gmail.com> Message-ID: <20140119200548.C18011A208F@async.async.caltech.edu> "less worried" = "I think it works"? I think the only threading that works in CM3 is user threading... I tried to run CM3-compiled binaries through "valgrind" but got an error that CM3 is using the same signal that valgrind does internally (on AMD64_LINUX at least). I think this is easy to change in m3core (yes?) but haven't gotten around to it... valgrind has a threading checker called "helgrind"... Mika Jay writes: > >--Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A >Content-Type: text/plain; > charset=utf-8 >Content-Transfer-Encoding: quoted-printable > >For native I'm less worried now. For wow64 I'm still worried & hope to follo= >w up further. > > - Jay > >On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" wrote:= > > >> Jay: >> =20 >> I=E2=80=99m very concerned about the threading not working properly on bot= >h 32-bit and 64-bit Windows. >> =20 >> The Thread Test program crashes for me on both platforms. >> =20 >> I haven=E2=80=99t tried your new test program yet. >> =20 >> --Randy >> =20 >> From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K >> Sent: Saturday, January 18, 2014 1:23 AM >> To: m3devel >> Subject: EXT:[M3devel] Win32 SuspendThread? >> =20 >> This program also doesn't behave as expected, native, nothing to do with w= >ow64.=20 >> Anyone else please confirm:=20 >> 1) my expectations -- it should never print anything >> 2) their importance -- garbage collector depends on it =20 >> 3) their not being met -- stuff gets printed=20 >> This is in the CVS repository, scratch/wow64stack/sync2.cpp=20 >> I am following up further.=20 >> Maybe we should get cooperative suspend really going?=20 >> Thank you. >> - Jay=20 >> =20 >> #include >> #include >> volatile long value; >> unsigned long __stdcall Thread(PVOID parameter) >> { >> while (1) >> InterlockedIncrement(&value); >> return 0; >> } >> int __cdecl main() >> { >> HANDLE thread =3D CreateThread(0, 0, Thread, 0, 0, 0); >> UINT i =3D 0; >> while (1) >> { >> i +=3D 1; >> if (SuspendThread(thread) =3D=3D (DWORD)-1) >> { >> printf("suspend failed %X\n", GetLastError()); >> Sleep(1); >> continue; >> } >> volatile long a =3D value;=20 >> volatile long b =3D value; >> if (a !=3D b) >> { >> printf("%d %d %d %d\n", i, a, b, b - a); >> } >> ResumeThread(thread); >> } >> } > >--Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A >Content-Type: text/html; > charset=utf-8 >Content-Transfer-Encoding: quoted-printable > >utf-8">
For native I'm less worried now. For w= >ow64 I'm still worried & hope to follow up further.

 - Jaydiv>

On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" <mailto:rcolebur at SCIRES.COM">rcolebur at SCIRES.COM> wrote:

= >
> >= > > > > > >
>

ibri","sans-serif";color:#1F497D">Jay:

>

ibri","sans-serif";color:#1F497D"> > >

ibri","sans-serif";color:#1F497D">I=E2=80=99m very concerned a= >bout the threading not working properly on both 32-bit and 64-bit Windows.:p>

>

ibri","sans-serif";color:#1F497D"> > >

ibri","sans-serif";color:#1F497D">The Thread Test program cra= >shes for me on both platforms.

>

ibri","sans-serif";color:#1F497D"> > >

ibri","sans-serif";color:#1F497D">I haven=E2=80=99t tried you= >r new test program yet.

>

ibri","sans-serif";color:#1F497D"> > >

ibri","sans-serif";color:#1F497D">--Randyp> >

ibri","sans-serif";color:#1F497D"> > >

>
in 0in"> >

e:10.0pt;font-family:"Tahoma","sans-serif"">From:= >s-serif""> jayk123 at hotmail.coma> [mailto:jayk123 at hotmail.com] >On Behalf Of Jay K
>Sent: Saturday, January 18, 2014 1:23 AM
>To: m3devel
>Subject: EXT:[M3devel] Win32 SuspendThread?

>
>
>

 

>
>

in-bottom:12.0pt;margin-left:.5in"> > = >;This program also doesn't behave as expected, native, nothing to do with wo= >w64. >
> Anyone else please confirm:
>   1) my expectations -- it should never print anything
>   2) their importance  -- garbage collector depends on it&nb= >sp;
>   3) their not being met -- stuff gets printed
> This is in the CVS repository, scratch/wow64stack/sync2.cpp
> I am following up further.
> Maybe we should get cooperative suspend really going?
>Thank you.
> - Jay

>#include <stdio.h>
>#include <windows.h>
>volatile long value;
>unsigned long __stdcall Thread(PVOID parameter)
>{
>  while (1)
>   InterlockedIncrement(&value);
>  return 0;
>}
>int __cdecl main()
>{
>  HANDLE thread =3D CreateThread(0, 0, Thread, 0, 0, 0);
>  UINT i =3D 0;
>  while (1)
>  {
>    i +=3D 1;
>    if (SuspendThread(thread) =3D=3D (DWORD)-1)
>    {
>      printf("suspend failed %X\n", GetLastError())= >;
>      Sleep(1);
>      continue;
>    }
>    volatile long a =3D value;
>    volatile long b =3D value;
>    if (a !=3D b)
>    {
>     printf("%d %d %d %d\n", i, a, b, b - a);
>    }
>    ResumeThread(thread);
>  }
>}

>
>
> > >
= > >--Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A-- From mika at async.caltech.edu Sun Jan 19 21:13:08 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sun, 19 Jan 2014 12:13:08 -0800 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: References: <52DBA2EF.4020101@marino.st>, , <52DBC081.1070401@marino.st> , <52DBC748.5090308@marino.st> Message-ID: <20140119201308.0A4B41A208F@async.async.caltech.edu> There were some bugs we never got around to addressing with the C backend, weren't there? A crash that didn't happen with native backend. I've been using the "native" at more or less the head on very recent Debian and the Raspberry Pi for a while now with no problems to speak of. (As long as you only use user threads... but that's another story, I think.) Mika Jay K writes: >--_f56ed036-02b3-494e-bb04-f7c58d07fa28_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > >I understand your reluctance. > > >> Is the goal just to get FreeBSD to have a binary with the latest >> sources? > > >Yes=2C and to make sure the build process works for you. > > >> Shouldn't FreeBSD use the C backend too? > > >Every target should. I haven't convinced everyone yet. >There is less resistance for new targets=2C like AMD64_NT and ARM_LINUX=2C = >which are working=2C using the C backend. >There are downsides to the C backend: > - it is slower to compiler =20 > - debugging is degraded on systems that have m3gdb =20 > (debugging is much better on systems w/o m3gdb)=20 > > >And there are advantages not yet implemented=2C in particular=2C efficient = >exception handling by generating C++. > > >> > gcc/amd64 works for the same >> > for Linux=2C FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly. >>=20 >> I assume you mean after the system understands this target. There will >> be a lot of files with __FreeBSD__ that need to be changed to >> __FreeBSD__ || __DragonFly__ as well. In any case=2C I'll have to >> regenerate a bunch of patches from my current work. > > >Your patches should apply fairly cleanly to any new checkout. >I am willing to through them and apply them to CVS if you want -- depends o= >n if you are ok with my stealing your credit=2C vs. you want to commit them= > yourselves. >I already skimmed them and they all look easy. >DragonflyBSD is basically FreeBSD by another name. >Yes=2C a ton of work has been done on the kernel=2C file systems=2C maybe p= >orts. >But the ABI is almost the same -- heck=2C for the most part=2C Linux=3D=3DN= >etBSD=3D=3DFreeBSD=3D=3DOpenBSD. >Sure=2C they might be implemented differently=2C but they are highly source= > compatible and every highly object code compatible. > > >> I must be misunderstanding what you mean by "upgrade". It's the >> cross-compiler that's complaining=2C and it was built by the latest patch= >es. > >Agreed=2C we are both missing something simple. >Usually this is the error you get when you don't upgrade Target.i3/Target.m= >3. >If you want to press on with the current approach and ignore my advise=2C i= >t should be possible to step through all this. >To start=2C break on Target__Init and see if it returns true or false. > > >> statically while a bug in the cm3 sources caused those libraries to link >> dynamically (which is why release binaries don't run on current FreeBSD) > > >Ugh. I saw something about that in your diffs. Definitely we intend to link= > statically. >I actually went to the trouble of removing use of gmp/mpfr/mpc in current s= >ource. >They are used for actually very little and aren't worth it. > > > - Jay > > >> Date: Sun=2C 19 Jan 2014 13:38:32 +0100 >> From: adacore at marino.st >> To: jay.krell at cornell.edu >> CC: m3devel at elegosoft.com >> Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) >>=20 >> On 1/19/2014 13:17=2C Jay K wrote: >> > Hey..I like that that does capture some of My Way.=20 >> > Here is my independent off the cuff write up though: >> > I know the ports system is nice. >> > I know we really need to be in it for many operating systems. >> > I suspect your problem is that you didn't "upgrade" to apply you >> > changes at the right time. >>=20 >> I must be misunderstanding what you mean by "upgrade". It's the >> cross-compiler that's complaining=2C and it was built by the latest patch= >es. >>=20 >>=20 >> > Do any of these work for you: >> > https://modula3.elegosoft.com/cm3/snaps/snapshot-index.html >>=20 >> I'm not clear to which set you are referring. >> There's no need for a binary set=2C modula3 now builds on FreeBSD 9 and >> FreeBSD 10 (http://www.freshports.org/lang/modula3)=2C which is also >> superior to the release binaries because they link gmp and mpfr >> statically while a bug in the cm3 sources caused those libraries to link >> dynamically (which is why release binaries don't run on current FreeBSD) >>=20 >> Then there are the source distributions=2C but they are from 2012 so they >> aren't exactly fresh either. >>=20 >>=20 >> > Anyway=2C please try this:=20 >> > Ignore DragonFly at first.=20 >> > Go to a system with a working cm3. Such as FreeBSD. Linux. Almost >> > anything.=20 >> > Such as from ports.=20 >> > Make a backup=2C e.g. of /usr/local/cm3.=20 >> > My instructions are unfortunately destructive of the existing install= >.=20 >> > Checkout the full current CVS repository.=20 >> > In that repository=2C run scripts/python/upgrade.py.=20 >> > If that doesn't work=2C stop=2C and we'll help.=20 >>=20 >> Is the goal just to get FreeBSD to have a binary with the latest >> sources? If so=2C I'd just modify the port to do that now that we have >> working binary bootstraps. The main reason I'm reluctant is that it >> took over a week to make the cm3 port=2C I've never had so much difficult= >y >> before. (I've also spent another week on the DragonFly port). So I'm >> "resisting" because of how much work it already took. That's why I was >> trying to port DragonFly=2C THEN update to latest rather than update to >> latest and then port DragonFly. (in other words=2C going forward seems a >> short path than backtracking) >>=20 >>=20 >>=20 >> > If that does work=2C apply your diffs. And upgrade.py again. You can = >say >> > "upgrade.py skipgcc" here.=20 >> > Then boot1.py c amd64_dragonflybsd =20 >> > Make sure you say "c" and the target "amd64_dragonflybsd"=2C anywhe= >re >> > on the command line.=20 >> > Ok=2C "c" is actually up to you. Since nobody messes with ABI much= >=2C >> > gcc/amd64 works for the same >> > for Linux=2C FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly. >>=20 >> I assume you mean after the system understands this target. There will >> be a lot of files with __FreeBSD__ that need to be changed to >> __FreeBSD__ || __DragonFly__ as well. In any case=2C I'll have to >> regenerate a bunch of patches from my current work. >>=20 >>=20 >> > If this works=2C it will give you=2C roughly amd64_dragonflybsd*.tar.= >gz=20 >> > Copy that to your Dragonfly system=2C extract it=2C cd into it=2C mak= >e.=20 >> > If that works=2C you have a native cm3.=20 >> > Run it. It should say "can't find cm3.cfg". =20 >> > Now on Dragonfly=2C mkdir -p /usr/local/cm3/bin=20 >> > cp cm3 /usr/local/cm3/bin =20 >> > export PATH=3D/usr/local/cm3/bin:$PATH >> > full CVS checkout =20 >> > cd scripts/python =3B ./boot2.sh=20 >> > =20 >> > You will need to have made small changes to pylib.py for this to work= >. >> > (And your config file should say to use the C backend=2C like Darwin = >and >> > ARM_LINUX do.) >>=20 >>=20 >> Well=2C I'm thrilled if I don't have to patch gcc again=2C that takes a l= >ong >> time. Shouldn't FreeBSD use the C backend too? >>=20 >> John > = > >--_f56ed036-02b3-494e-bb04-f7c58d07fa28_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > >
I understand your reluctance.>

>=3B Is the goal just to get FreeBSD to have a binary with the l= >atest
>=3B sources?


Yes=2C and to make sure the build proce= >ss works for you.


>=3B Shouldn't FreeBSD use the C backend too= >?


Every target should. I haven't convinced everyone yet.
Ther= >e is less resistance for new targets=2C like AMD64_NT and ARM_LINUX=2C whic= >h are working=2C using the C backend.
There are downsides to the C backe= >nd:
 =3B - it is slower to compiler =3B
 =3B - debugging= > is degraded on systems that have m3gdb =3B
 =3B(debugging is m= >uch better on systems w/o m3gdb)


And there are advantages not y= >et implemented=2C in particular=2C efficient exception handling by generati= >ng C++.


>=3B >=3B gcc/amd64 works for the same
>=3B >= >=3B for Linux=2C FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.
>= >=3B
>=3B I assume you mean after the system understands this target. = > There will
>=3B be a lot of files with __FreeBSD__ that need to be ch= >anged to
>=3B __FreeBSD__ || __DragonFly__ as well. In any case=2C I'= >ll have to
>=3B regenerate a bunch of patches from my current work.>

Your patches should apply fairly cleanly to any new checkout.
I= > am willing to through them and apply them to CVS if you want -- depends on= > if you are ok with my stealing your credit=2C vs. you want to commit them = >yourselves.
I already skimmed them and they all look easy.
DragonflyB= >SD is basically FreeBSD by another name.
Yes=2C a ton of work has been d= >one on the kernel=2C file systems=2C maybe ports.
But the ABI is almost = >the same -- heck=2C for the most part=2C Linux=3D=3DNetBSD=3D=3DFreeBSD=3D= >=3DOpenBSD.
Sure=2C they might be implemented differently=2C but they ar= >e highly source compatible and every highly object code compatible.

= >
>=3B I must be misunderstanding what you mean by "upgrade". It's the= >
>=3B cross-compiler that's complaining=2C and it was built by the lat= >est patches.

Agreed=2C we are both missing something simple.
Usua= >lly this is the error you get when you don't upgrade Target.i3/Target.m3.r>If you want to press on with the current approach and ignore my advise=2C= > it should be possible to step through all this.
To start=2C break on Ta= >rget__Init and see if it returns true or false.


>=3B staticall= >y while a bug in the cm3 sources caused those libraries to link
>=3B d= >ynamically (which is why release binaries don't run on current FreeBSD)
= >

Ugh. I saw something about that in your diffs. Definitely we intend= > to link statically.
I actually went to the trouble of removing use of g= >mp/mpfr/mpc in current source.
They are used for actually very little an= >d aren't worth it.


 =3B- Jay


>=3B Date: Su= >n=2C 19 Jan 2014 13:38:32 +0100
>=3B From: adacore at marino.st
>=3B= > To: jay.krell at cornell.edu
>=3B CC: m3devel at elegosoft.com
>=3B Su= >bject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
&g= >t=3B
>=3B On 1/19/2014 13:17=2C Jay K wrote:
>=3B >=3B Hey..I = >like that that does capture some of My Way.
>=3B >=3B Here is my in= >dependent off the cuff write up though:
>=3B >=3B I know the ports = >system is nice.
>=3B >=3B I know we really need to be in it for man= >y operating systems.
>=3B >=3B I suspect your problem is that you d= >idn't "upgrade" to apply you
>=3B >=3B changes at the right time.>>=3B
>=3B I must be misunderstanding what you mean by "upgrade". = >It's the
>=3B cross-compiler that's complaining=2C and it was built by= > the latest patches.
>=3B
>=3B
>=3B >=3B Do any of thes= >e work for you:
>=3B >=3B https://modula3.elegosoft.com/cm3/snaps/s= >napshot-index.html
>=3B
>=3B I'm not clear to which set you are = >referring.
>=3B There's no need for a binary set=2C modula3 now builds= > on FreeBSD 9 and
>=3B FreeBSD 10 (http://www.freshports.org/lang/modu= >la3)=2C which is also
>=3B superior to the release binaries because th= >ey link gmp and mpfr
>=3B statically while a bug in the cm3 sources ca= >used those libraries to link
>=3B dynamically (which is why release bi= >naries don't run on current FreeBSD)
>=3B
>=3B Then there are th= >e source distributions=2C but they are from 2012 so they
>=3B aren't e= >xactly fresh either.
>=3B
>=3B
>=3B >=3B Anyway=2C plea= >se try this:
>=3B >=3B Ignore DragonFly at first.
>=3B >= >=3B Go to a system with a working cm3. Such as FreeBSD. Linux. Almost
= >>=3B >=3B anything.
>=3B >=3B Such as from ports.
>=3B = >>=3B Make a backup=2C e.g. of /usr/local/cm3.
>=3B >=3B My in= >structions are unfortunately destructive of the existing install.
>= >=3B >=3B Checkout the full current CVS repository.
>=3B >=3B = >In that repository=2C run scripts/python/upgrade.py.
>=3B >=3B If= > that doesn't work=2C stop=2C and we'll help.
>=3B
>=3B Is the = >goal just to get FreeBSD to have a binary with the latest
>=3B sources= >? If so=2C I'd just modify the port to do that now that we have
>=3B = >working binary bootstraps. The main reason I'm reluctant is that it
>= >=3B took over a week to make the cm3 port=2C I've never had so much difficu= >lty
>=3B before. (I've also spent another week on the DragonFly port)= >. So I'm
>=3B "resisting" because of how much work it already took. = >That's why I was
>=3B trying to port DragonFly=2C THEN update to lates= >t rather than update to
>=3B latest and then port DragonFly. (in othe= >r words=2C going forward seems a
>=3B short path than backtracking)>>=3B
>=3B
>=3B
>=3B >=3B If that does work=2C appl= >y your diffs. And upgrade.py again. You can say
>=3B >=3B "upgrade.p= >y skipgcc" here.
>=3B >=3B Then boot1.py c amd64_dragonflybsd <= >br>>=3B >=3B Make sure you say "c" and the target "amd64_dragonflyb= >sd"=2C anywhere
>=3B >=3B on the command line.
>=3B >=3B = > Ok=2C "c" is actually up to you. Since nobody messes with ABI much=2C
&= >gt=3B >=3B gcc/amd64 works for the same
>=3B >=3B for Linux=2C= > FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.
>=3B
>=3B I assu= >me you mean after the system understands this target. There will
>=3B= > be a lot of files with __FreeBSD__ that need to be changed to
>=3B __= >FreeBSD__ || __DragonFly__ as well. In any case=2C I'll have to
>=3B = >regenerate a bunch of patches from my current work.
>=3B
>=3B r>>=3B >=3B If this works=2C it will give you=2C roughly amd64_dragon= >flybsd*.tar.gz
>=3B >=3B Copy that to your Dragonfly system=2C ex= >tract it=2C cd into it=2C make.
>=3B >=3B If that works=2C you ha= >ve a native cm3.
>=3B >=3B Run it. It should say "can't find cm3.= >cfg".
>=3B >=3B Now on Dragonfly=2C mkdir -p /usr/local/cm3/bin = >
>=3B >=3B cp cm3 /usr/local/cm3/bin
>=3B >=3B export P= >ATH=3D/usr/local/cm3/bin:$PATH
>=3B >=3B full CVS checkout
&g= >t=3B >=3B cd scripts/python =3B ./boot2.sh
>=3B >=3B
>= >=3B >=3B You will need to have made small changes to pylib.py for this = >to work.
>=3B >=3B (And your config file should say to use the C b= >ackend=2C like Darwin and
>=3B >=3B ARM_LINUX do.)
>=3B
>= >=3B
>=3B Well=2C I'm thrilled if I don't have to patch gcc again=2C t= >hat takes a long
>=3B time. Shouldn't FreeBSD use the C backend too?<= >br>>=3B
>=3B John
>= > >--_f56ed036-02b3-494e-bb04-f7c58d07fa28_-- From mika at async.caltech.edu Sun Jan 19 22:36:49 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sun, 19 Jan 2014 13:36:49 -0800 Subject: [M3devel] narrow still failing.. Message-ID: <20140119213649.403AE1A208F@async.async.caltech.edu> A smaller test: 1 MODULE Main; 2 IMPORT BDD, IO; 3 4 PROCEDURE P() : REFANY = 5 BEGIN 6 VAR x : REFANY; 7 BEGIN 8 x := BDD.New("a"); 9 Z(x); 10 RETURN x 11 END 12 END P; 13 14 PROCEDURE Q() = 15 BEGIN 16 IO.Put(P() & "\n") 17 END Q; 18 19 PROCEDURE Z(VAR x : REFANY) = 20 BEGIN 21 x := BDD.Format(x) 22 END Z; 23 24 BEGIN 25 Q() 26 END Main. (191)rover:~/refany/src>../FreeBSD4/refany *** *** runtime error: *** NARROW failed *** file "/big/home/mika/refany/src/Main.m3", line 21 *** What's going on here??? Here are the relevant declarations: (* simple BDD package *) INTERFACE BDD; IMPORT Word; TYPE T <: REFANY; (* the boolean constants top and bottom *) PROCEDURE True() : T; PROCEDURE False() : T; (* a new variable *) PROCEDURE New(name : TEXT := NIL) : T; (* unary ops *) PROCEDURE Not(a : T) : T; (* binary ops *) PROCEDURE And(a, b : T) : T; PROCEDURE Xor(a , b : T) : T; PROCEDURE Equivalent(a, b : T) : T; PROCEDURE Or( a, b : T) : T; PROCEDURE Implies (a , b : T) : T; (* maketrue/makefalse *) PROCEDURE MakeFalse(b, v : T) : T; (* make v false in b *) PROCEDURE MakeTrue(b, v : T) : T; (* make v true in b *) (* print with ids *) PROCEDURE Format(a : T; symtab : REFANY (* BDDTextTbl.T *) := NIL) : TEXT; (* the following procedures allow this interface to be used in generics *) PROCEDURE Equal(a, b : T) : BOOLEAN; PROCEDURE Hash(a : T) : Word.T; PROCEDURE Size(a : T) : CARDINAL; (* number of nodes in structure *) CONST Brand = "BDD 0.1"; PROCEDURE GetId(a : T) : INTEGER; (* for debugging *) END BDD. .... REVEAL T = BRANDED Brand OBJECT l , r : T; root : Root; tag : CARDINAL; (* for hashing *) name : TEXT; METHODS init() : T := Init; END; PROCEDURE New(name : TEXT) : T = VAR res : Root := NEW(Root).init(); BEGIN res.name := name; res.root := res; res.l := true; res.r := false; res.id := nextId; res.tab := NEW(BDDTripleHash.Default).init(128); FOR i := FIRST(res.cache) TO LAST(res.cache) DO res.cache[i] := NEW(BDDTripleHash.Default).init(64) END; EVAL BDDTripleHash.Put(res.tab, Pair { res.l, res.r }, (res)); INC(nextId); RETURN res END New; PROCEDURE Format(x : T; symtab : REFANY := NIL) : TEXT = VAR nm : TEXT; BEGIN IF symtab # NIL AND NARROW(symtab, BDDTextTbl.T).get(x, nm) THEN RETURN nm END; IF x = true THEN RETURN "TRUE" ELSIF x = false THEN RETURN "FALSE" END; IF x.name # NIL THEN RETURN x.name END; RETURN Fmt.Int(x.root.id) & " && (" & Format(x.l) & ") || (" & Format(x.r) & ") && ~" & Fmt.Int(x.root.id) END Format; use option @M3stackdump to get a stack trace Abort I'm happy to share the whole package if anyone with a better clue than me would like a look. In fact: http://rover.gcapltd.com/~mika/bdd.tgz It should build without anything outside of standard cm3. Mika From mika at async.caltech.edu Sun Jan 19 23:01:23 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sun, 19 Jan 2014 14:01:23 -0800 Subject: [M3devel] narrow still failing.. In-Reply-To: <20140119213649.403AE1A208F@async.async.caltech.edu> References: <20140119213649.403AE1A208F@async.async.caltech.edu> Message-ID: <20140119220123.0D1971A208F@async.async.caltech.edu> As in the ISTYPE/TYPECASE example, the following works: PROCEDURE Z(VAR x : REFANY) = BEGIN TYPECASE x OF BDD.T(b) => x := BDD.Format(b) END; END Z; (244)rover:~/refany/src>../FreeBSD4/refany a This does NOT work: PROCEDURE Z(VAR x : REFANY) = BEGIN TYPECASE x OF BDD.T(b) => x := BDD.Format(x) END; END Z; (242)rover:~/refany/src>../FreeBSD4/refany *** *** runtime error: *** NARROW failed *** file "/big/home/mika/refany/src/Main.m3", line 22 *** use option @M3stackdump to get a stack trace Abort Same result PM3 / CM3. Mika From jay.krell at cornell.edu Sun Jan 19 23:58:27 2014 From: jay.krell at cornell.edu (Jay K) Date: Sun, 19 Jan 2014 22:58:27 +0000 Subject: [M3devel] Cooperative suspend? In-Reply-To: References: <8BC1682A-3760-44D0-BA67-3A0F3EA10E27@m3w.org> , Message-ID: 1) portability; We currently have separate code, roughly, for MacOSX, FreeBSD, OpenBSD, NT, and the other Posixes. MacOSX doesn't have Posix semaphores, for example. This would give us one codebase here. Only IA64 would likely remain an outlier, because stack is not merely the address of one local to another, but includes also the grow-up stack for the register saves. 2) It'd workaround the wow64 problem. ?- Jay ________________________________ > Subject: Re: [M3devel] Cooperative suspend? > From: dragisha at m3w.org > Date: Sun, 19 Jan 2014 15:28:18 +0100 > CC: m3devel at elegosoft.com > To: jay.krell at cornell.edu > > Why would we do this, and not signalling, as in POSIX implementation? > > Why try to make something synchronous when we have working asynchronous > implementation? > > Why would it be better than sync one? > > On 19 Jan 2014, at 13:00, Jay K > > wrote: > > There is a list of threads maintained by the m3 runtime. > Not by enumerating OS threads. > The thread structure contains in it perhaps a boolean or enum as to > suspend requested, suspending, suspended, resuming, running, etc. > From jay.krell at cornell.edu Mon Jan 20 00:13:24 2014 From: jay.krell at cornell.edu (Jay K) Date: Sun, 19 Jan 2014 23:13:24 +0000 Subject: [M3devel] Win32 SuspendThread? In-Reply-To: <20140119200548.C18011A208F@async.async.caltech.edu> References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL> <9EC3F463-F248-4C79-AF3F-130D2252B8B0@gmail.com>, <20140119200548.C18011A208F@async.async.caltech.edu> Message-ID: "not worried" for native windows. Still worried about wow64. The answer for native is in Rotor/sscli and I have a program to show it working/not-working. Specifically SuspendThread needs to be followed by GetThreadContext to ensure the suspend has completed. If we don't do that, we must. If we already do, we are ok. I'll look later. wow64 is still a problem. I'm still looking into it and thinking about it. But we do have native AMD64_NT so just saying wow64 doesn't work might be the answer. I know people will protest that it works, but doesn't always. I haven't figured out what rotor/sscli does. Java uses cooperative suspend, which sidesteps this stuff. ?- Jay ---------------------------------------- > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Win32 SuspendThread? > Date: Sun, 19 Jan 2014 12:05:48 -0800 > From: mika at async.caltech.edu > > "less worried" = "I think it works"? > > I think the only threading that works in CM3 is user threading... > > I tried to run CM3-compiled binaries through "valgrind" but got an > error that CM3 is using the same signal that valgrind does internally > (on AMD64_LINUX at least). I think this is easy to change in m3core > (yes?) but haven't gotten around to it... > > valgrind has a threading checker called "helgrind"... > > Mika > > Jay writes: >> >>--Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A >>Content-Type: text/plain; >> charset=utf-8 >>Content-Transfer-Encoding: quoted-printable >> >>For native I'm less worried now. For wow64 I'm still worried & hope to follo= >>w up further. >> >> - Jay >> >>On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" wrote:= >> >> >>> Jay: >>> =20 >>> I=E2=80=99m very concerned about the threading not working properly on bot= >>h 32-bit and 64-bit Windows. >>> =20 >>> The Thread Test program crashes for me on both platforms. >>> =20 >>> I haven=E2=80=99t tried your new test program yet. >>> =20 >>> --Randy >>> =20 >>> From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K >>> Sent: Saturday, January 18, 2014 1:23 AM >>> To: m3devel >>> Subject: EXT:[M3devel] Win32 SuspendThread? >>> =20 >>> This program also doesn't behave as expected, native, nothing to do with w= >>ow64.=20 >>> Anyone else please confirm:=20 >>> 1) my expectations -- it should never print anything >>> 2) their importance -- garbage collector depends on it =20 >>> 3) their not being met -- stuff gets printed=20 >>> This is in the CVS repository, scratch/wow64stack/sync2.cpp=20 >>> I am following up further.=20 >>> Maybe we should get cooperative suspend really going?=20 >>> Thank you. >>> - Jay=20 >>> =20 >>> #include >>> #include >>> volatile long value; >>> unsigned long __stdcall Thread(PVOID parameter) >>> { >>> while (1) >>> InterlockedIncrement(&value); >>> return 0; >>> } >>> int __cdecl main() >>> { >>> HANDLE thread =3D CreateThread(0, 0, Thread, 0, 0, 0); >>> UINT i =3D 0; >>> while (1) >>> { >>> i +=3D 1; >>> if (SuspendThread(thread) =3D=3D (DWORD)-1) >>> { >>> printf("suspend failed %X\n", GetLastError()); >>> Sleep(1); >>> continue; >>> } >>> volatile long a =3D value;=20 >>> volatile long b =3D value; >>> if (a !=3D b) >>> { >>> printf("%d %d %d %d\n", i, a, b, b - a); >>> } >>> ResumeThread(thread); >>> } >>> } >> >>--Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A >>Content-Type: text/html; >> charset=utf-8 >>Content-Transfer-Encoding: quoted-printable >> >>>utf-8">
For native I'm less worried now. For w= >>ow64 I'm still worried & hope to follow up further.

 - Jay>div>

On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" <>mailto:rcolebur at SCIRES.COM">rcolebur at SCIRES.COM> wrote:

= >>
>> >>= >> >> >> >>
I understand your reluctance.>>

>=3B Is the goal just to get FreeBSD to have a binary with the l= >>atest
>=3B sources?


Yes=2C and to make sure the build proce= >>ss works for you.


>=3B Shouldn't FreeBSD use the C backend too= >>?


Every target should. I haven't convinced everyone yet.
Ther= >>e is less resistance for new targets=2C like AMD64_NT and ARM_LINUX=2C whic= >>h are working=2C using the C backend.
There are downsides to the C backe= >>nd:
 =3B - it is slower to compiler =3B
 =3B - debugging= >> is degraded on systems that have m3gdb =3B
 =3B(debugging is m= >>uch better on systems w/o m3gdb)


And there are advantages not y= >>et implemented=2C in particular=2C efficient exception handling by generati= >>ng C++.


>=3B >=3B gcc/amd64 works for the same
>=3B >= >>=3B for Linux=2C FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.
>= >>=3B
>=3B I assume you mean after the system understands this target. = >> There will
>=3B be a lot of files with __FreeBSD__ that need to be ch= >>anged to
>=3B __FreeBSD__ || __DragonFly__ as well. In any case=2C I'= >>ll have to
>=3B regenerate a bunch of patches from my current work.>>

Your patches should apply fairly cleanly to any new checkout.
I= >> am willing to through them and apply them to CVS if you want -- depends on= >> if you are ok with my stealing your credit=2C vs. you want to commit them = >>yourselves.
I already skimmed them and they all look easy.
DragonflyB= >>SD is basically FreeBSD by another name.
Yes=2C a ton of work has been d= >>one on the kernel=2C file systems=2C maybe ports.
But the ABI is almost = >>the same -- heck=2C for the most part=2C Linux=3D=3DNetBSD=3D=3DFreeBSD=3D= >>=3DOpenBSD.
Sure=2C they might be implemented differently=2C but they ar= >>e highly source compatible and every highly object code compatible.

= >>
>=3B I must be misunderstanding what you mean by "upgrade". It's the= >>
>=3B cross-compiler that's complaining=2C and it was built by the lat= >>est patches.

Agreed=2C we are both missing something simple.
Usua= >>lly this is the error you get when you don't upgrade Target.i3/Target.m3.>r>If you want to press on with the current approach and ignore my advise=2C= >> it should be possible to step through all this.
To start=2C break on Ta= >>rget__Init and see if it returns true or false.


>=3B staticall= >>y while a bug in the cm3 sources caused those libraries to link
>=3B d= >>ynamically (which is why release binaries don't run on current FreeBSD)
= >>

Ugh. I saw something about that in your diffs. Definitely we intend= >> to link statically.
I actually went to the trouble of removing use of g= >>mp/mpfr/mpc in current source.
They are used for actually very little an= >>d aren't worth it.


 =3B- Jay


>=3B Date: Su= >>n=2C 19 Jan 2014 13:38:32 +0100
>=3B From: adacore at marino.st
>=3B= >> To: jay.krell at cornell.edu
>=3B CC: m3devel at elegosoft.com
>=3B Su= >>bject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
&g= >>t=3B
>=3B On 1/19/2014 13:17=2C Jay K wrote:
>=3B >=3B Hey..I = >>like that that does capture some of My Way.
>=3B >=3B Here is my in= >>dependent off the cuff write up though:
>=3B >=3B I know the ports = >>system is nice.
>=3B >=3B I know we really need to be in it for man= >>y operating systems.
>=3B >=3B I suspect your problem is that you d= >>idn't "upgrade" to apply you
>=3B >=3B changes at the right time.>>>=3B
>=3B I must be misunderstanding what you mean by "upgrade". = >>It's the
>=3B cross-compiler that's complaining=2C and it was built by= >> the latest patches.
>=3B
>=3B
>=3B >=3B Do any of thes= >>e work for you:
>=3B >=3B https://modula3.elegosoft.com/cm3/snaps/s= >>napshot-index.html
>=3B
>=3B I'm not clear to which set you are = >>referring.
>=3B There's no need for a binary set=2C modula3 now builds= >> on FreeBSD 9 and
>=3B FreeBSD 10 (http://www.freshports.org/lang/modu= >>la3)=2C which is also
>=3B superior to the release binaries because th= >>ey link gmp and mpfr
>=3B statically while a bug in the cm3 sources ca= >>used those libraries to link
>=3B dynamically (which is why release bi= >>naries don't run on current FreeBSD)
>=3B
>=3B Then there are th= >>e source distributions=2C but they are from 2012 so they
>=3B aren't e= >>xactly fresh either.
>=3B
>=3B
>=3B >=3B Anyway=2C plea= >>se try this:
>=3B >=3B Ignore DragonFly at first.
>=3B >= >>=3B Go to a system with a working cm3. Such as FreeBSD. Linux. Almost
= >>>=3B >=3B anything.
>=3B >=3B Such as from ports.
>=3B = >>>=3B Make a backup=2C e.g. of /usr/local/cm3.
>=3B >=3B My in= >>structions are unfortunately destructive of the existing install.
>= >>=3B >=3B Checkout the full current CVS repository.
>=3B >=3B = >>In that repository=2C run scripts/python/upgrade.py.
>=3B >=3B If= >> that doesn't work=2C stop=2C and we'll help.
>=3B
>=3B Is the = >>goal just to get FreeBSD to have a binary with the latest
>=3B sources= >>? If so=2C I'd just modify the port to do that now that we have
>=3B = >>working binary bootstraps. The main reason I'm reluctant is that it
>= >>=3B took over a week to make the cm3 port=2C I've never had so much difficu= >>lty
>=3B before. (I've also spent another week on the DragonFly port)= >>. So I'm
>=3B "resisting" because of how much work it already took. = >>That's why I was
>=3B trying to port DragonFly=2C THEN update to lates= >>t rather than update to
>=3B latest and then port DragonFly. (in othe= >>r words=2C going forward seems a
>=3B short path than backtracking)>>>=3B
>=3B
>=3B
>=3B >=3B If that does work=2C appl= >>y your diffs. And upgrade.py again. You can say
>=3B >=3B "upgrade.p= >>y skipgcc" here.
>=3B >=3B Then boot1.py c amd64_dragonflybsd <= >>br>>=3B >=3B Make sure you say "c" and the target "amd64_dragonflyb= >>sd"=2C anywhere
>=3B >=3B on the command line.
>=3B >=3B = >> Ok=2C "c" is actually up to you. Since nobody messes with ABI much=2C
&= >>gt=3B >=3B gcc/amd64 works for the same
>=3B >=3B for Linux=2C= >> FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.
>=3B
>=3B I assu= >>me you mean after the system understands this target. There will
>=3B= >> be a lot of files with __FreeBSD__ that need to be changed to
>=3B __= >>FreeBSD__ || __DragonFly__ as well. In any case=2C I'll have to
>=3B = >>regenerate a bunch of patches from my current work.
>=3B
>=3B >r>>=3B >=3B If this works=2C it will give you=2C roughly amd64_dragon= >>flybsd*.tar.gz
>=3B >=3B Copy that to your Dragonfly system=2C ex= >>tract it=2C cd into it=2C make.
>=3B >=3B If that works=2C you ha= >>ve a native cm3.
>=3B >=3B Run it. It should say "can't find cm3.= >>cfg".
>=3B >=3B Now on Dragonfly=2C mkdir -p /usr/local/cm3/bin = >>
>=3B >=3B cp cm3 /usr/local/cm3/bin
>=3B >=3B export P= >>ATH=3D/usr/local/cm3/bin:$PATH
>=3B >=3B full CVS checkout
&g= >>t=3B >=3B cd scripts/python =3B ./boot2.sh
>=3B >=3B
>= >>=3B >=3B You will need to have made small changes to pylib.py for this = >>to work.
>=3B >=3B (And your config file should say to use the C b= >>ackend=2C like Darwin and
>=3B >=3B ARM_LINUX do.)
>=3B
>= >>=3B
>=3B Well=2C I'm thrilled if I don't have to patch gcc again=2C t= >>hat takes a long
>=3B time. Shouldn't FreeBSD use the C backend too?<= >>br>>=3B
>=3B John
>>= >> >>--_f56ed036-02b3-494e-bb04-f7c58d07fa28_-- From mika at async.caltech.edu Mon Jan 20 00:39:38 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sun, 19 Jan 2014 15:39:38 -0800 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: References: <52DBA2EF.4020101@marino.st>, , <52DBC081.1070401@marino.st> , <52DBC748.5090308@marino.st> , <20140119201308.0A4B41A208F@async.async.caltech.edu> Message-ID: <20140119233938.9C5A31A208F@async.async.caltech.edu> Jay K writes: >The only problem I remember is an alignment matter using msvcrt.dll on AMD6= >4_NT.=0A= >I'll look into it=2C and the old emails.=0A= >I've used the C backend on many platforms now.=0A= This stuff doesn't happen with the native backend: > 2. I can't get anything with Trestle working. I'm running in a VNC > session on a remote machine. Not sure this has anything to do with the > compiler or even the ARM. Has anyone seen it before (on any system)? > > (gdb) cont > Continuing. > [New Thread 0x42276430 (LWP 24451)] > [Thread 0x42276430 (LWP 24451) exited] > [New Thread 0x424c6430 (LWP 24454)] > > > *** > *** runtime error: > *** Exception "ZChildVBT.BadPercentage" not in RAISES list > *** file "../src/lego/ZChildVBT.m3", line 61 > *** > > > Program received signal SIGABRT, Aborted. > > backtrace: > > (gdb) where > #0 ZChildVBT__Init (z_L_120=0x422bbf70 "H\357K", ch_L_121=0x422bdb84 "\260\326K", h_L_122=0, v_L_123=1.75, loc_L_124=4 '\004', type_L_125=1 '\001', shaper_ L_126=0x41a32b54, open_L_127=0 '\000') at ../src/lego/ZCh > ildVBT.m3:61 > #1 0x40637518 in FormsVBT__pZChild (cl_L_1042=0x42280444, list_L_1043=0x424c54c4, s_L_1044=0x424c55dc) at ../src/FormsVBT.m3:2593 > #2 0x40609f30 in FormsVBT__Item (cl_L_301=0x42280444, exp_L_302=0x41a45994 "", state_L_303=0x424c55dc) at ../src/FormsVBT.m3:250 > #3 0x40648b3c in FormsVBT__AddChildren (cl_L_1229=0x42280444, v_L_1230=0x422b0634 " \365K", list_L_1231=0x41a48504, state_L_1232=0x424c55dc) at ../src/FormsVBT.m3:3671 > #4 0x40634138 in FormsVBT__pZSplit (cl_L_999=0x42280444, list_L_1000=0x424c56bc, s_L_1001=0x424c57c4) at ../src/FormsVBT.m3:2454 > #5 0x40609f30 in FormsVBT__Item (cl_L_301=0x42280444, exp_L_302=0x41a40ec0 "", state_L_303=0x424c57c4) at ../src/FormsVBT.m3:250 > #6 0x4064838c in FormsVBT__OneChild (cl_L_1224=0x42280444, list_L_1225=0x0, state_L_1226=0x424c57c4) at ../src/FormsVBT.m3:3642 > #7 0x406129c0 in FormsVBT__pShape (cl_L_496=0x42280444, list_L_497=0x424c58a4, s_L_498=0x424c59a4) at ../src/FormsVBT.m3:948 > #8 0x40609f30 in FormsVBT__Item (cl_L_301=0x42280444, exp_L_302=0x41a402a4 "", state_L_303=0x424c59a4) at ../src/FormsVBT.m3:250 > #9 0x4064838c in FormsVBT__OneChild (cl_L_1224=0x42280444, list_L_1225=0x0, state_L_1226=0x424c59a4) at ../src/FormsVBT.m3:3642 > #10 0x4062ca10 in FormsVBT__pScale (cl_L_905=0x42280444, list_L_906=0x424c5a84, s_L_907=0x42280458) at ../src/FormsVBT.m3:2165 > #11 0x40609f30 in FormsVBT__Item (cl_L_301=0x42280444, exp_L_302=0x41a4006c "", state_L_303=0x42280458) at ../src/FormsVBT.m3:250 > #12 0x40606dec in FormsVBT__Apply (cl_L_293=0x42280444) at ../src/FormsVBT.m3:84 > #13 0x40d27bd4 in ThreadPThread__RunThread (me_L_231=0x4c5d78 "0SLB\330]L") at ../src/thread/PTHREAD/ThreadPThread.m3:449 > #14 0x40d27590 in ThreadPThread__ThreadBase (param_L_229=0x4c5d78 "0SLB\330]L") at ../src/thread/PTHREAD/ThreadPThread.m3:422 > #15 0x41850bfc in start_thread () from /lib/arm-linux-gnueabihf/libpthread.so.0 > #16 0x41933758 in ?? () from /lib/arm-linux-gnueabihf/libc.so.6 > #17 0x41933758 in ?? () from /lib/arm-linux-gnueabihf/libc.so.6 > Backtrace stopped: previous frame identical to this frame (corrupt stack?) > (gdb) cont > Continuing. > From jay.krell at cornell.edu Mon Jan 20 01:06:27 2014 From: jay.krell at cornell.edu (Jay) Date: Sun, 19 Jan 2014 16:06:27 -0800 Subject: [M3devel] Win32 SuspendThread? In-Reply-To: <20140119200548.C18011A208F@async.async.caltech.edu> References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL> <9EC3F463-F248-4C79-AF3F-130D2252B8B0@gmail.com> <20140119200548.C18011A208F@async.async.caltech.edu> Message-ID: Cooperative suspend will also fix this signal allocation vs. valgrind problem. Can also likely pick a different one... - Jay On Jan 19, 2014, at 12:05 PM, wrote: > "less worried" = "I think it works"? > > I think the only threading that works in CM3 is user threading... > > I tried to run CM3-compiled binaries through "valgrind" but got an > error that CM3 is using the same signal that valgrind does internally > (on AMD64_LINUX at least). I think this is easy to change in m3core > (yes?) but haven't gotten around to it... > > valgrind has a threading checker called "helgrind"... > > Mika > > Jay writes: >> >> --Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A >> Content-Type: text/plain; >> charset=utf-8 >> Content-Transfer-Encoding: quoted-printable >> >> For native I'm less worried now. For wow64 I'm still worried & hope to follo= >> w up further. >> >> - Jay >> >> On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" wrote:= >> >> >>> Jay: >>> =20 >>> I=E2=80=99m very concerned about the threading not working properly on bot= >> h 32-bit and 64-bit Windows. >>> =20 >>> The Thread Test program crashes for me on both platforms. >>> =20 >>> I haven=E2=80=99t tried your new test program yet. >>> =20 >>> --Randy >>> =20 >>> From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K >>> Sent: Saturday, January 18, 2014 1:23 AM >>> To: m3devel >>> Subject: EXT:[M3devel] Win32 SuspendThread? >>> =20 >>> This program also doesn't behave as expected, native, nothing to do with w= >> ow64.=20 >>> Anyone else please confirm:=20 >>> 1) my expectations -- it should never print anything >>> 2) their importance -- garbage collector depends on it =20 >>> 3) their not being met -- stuff gets printed=20 >>> This is in the CVS repository, scratch/wow64stack/sync2.cpp=20 >>> I am following up further.=20 >>> Maybe we should get cooperative suspend really going?=20 >>> Thank you. >>> - Jay=20 >>> =20 >>> #include >>> #include >>> volatile long value; >>> unsigned long __stdcall Thread(PVOID parameter) >>> { >>> while (1) >>> InterlockedIncrement(&value); >>> return 0; >>> } >>> int __cdecl main() >>> { >>> HANDLE thread =3D CreateThread(0, 0, Thread, 0, 0, 0); >>> UINT i =3D 0; >>> while (1) >>> { >>> i +=3D 1; >>> if (SuspendThread(thread) =3D=3D (DWORD)-1) >>> { >>> printf("suspend failed %X\n", GetLastError()); >>> Sleep(1); >>> continue; >>> } >>> volatile long a =3D value;=20 >>> volatile long b =3D value; >>> if (a !=3D b) >>> { >>> printf("%d %d %d %d\n", i, a, b, b - a); >>> } >>> ResumeThread(thread); >>> } >>> } >> >> --Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A >> Content-Type: text/html; >> charset=utf-8 >> Content-Transfer-Encoding: quoted-printable >> >> > utf-8">
For native I'm less worried now. For w= >> ow64 I'm still worried & hope to follow up further.

 - Jay> div>

On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" <> mailto:rcolebur at SCIRES.COM">rcolebur at SCIRES.COM> wrote:

= >>
>> >> = >> >> >> >> >> >>
>>

> ibri","sans-serif";color:#1F497D">Jay:

>>

> ibri","sans-serif";color:#1F497D"> >

> ibri","sans-serif";color:#1F497D">I=E2=80=99m very concerned a= >> bout the threading not working properly on both 32-bit and 64-bit Windows.> :p>

>>

> ibri","sans-serif";color:#1F497D"> >

> ibri","sans-serif";color:#1F497D">The Thread Test program cra= >> shes for me on both platforms.

>>

> ibri","sans-serif";color:#1F497D"> >

> ibri","sans-serif";color:#1F497D">I haven=E2=80=99t tried you= >> r new test program yet.

>>

> ibri","sans-serif";color:#1F497D"> >

> ibri","sans-serif";color:#1F497D">--Randy> p> >>

> ibri","sans-serif";color:#1F497D"> >

>>
> in 0in"> >>

> e:10.0pt;font-family:"Tahoma","sans-serif"">From:= >> > s-serif""> jayk123 at hotmail.com> a> [mailto:jayk123 at hotmail.com] >> On Behalf Of Jay K
>> Sent: Saturday, January 18, 2014 1:23 AM
>> To: m3devel
>> Subject: EXT:[M3devel] Win32 SuspendThread?

>>
>>
>>

 

>>
>>

> in-bottom:12.0pt;margin-left:.5in"> >>  = >> ;This program also doesn't behave as expected, native, nothing to do with wo= >> w64. >>
>>  Anyone else please confirm:
>>    1) my expectations -- it should never print anything
>>    2) their importance  -- garbage collector depends on it&nb= >> sp;
>>    3) their not being met -- stuff gets printed
>>  This is in the CVS repository, scratch/wow64stack/sync2.cpp
>>  I am following up further.
>>  Maybe we should get cooperative suspend really going?
>> Thank you.
>>  - Jay
>>  
>> #include <stdio.h>
>> #include <windows.h>
>> volatile long value;
>> unsigned long __stdcall Thread(PVOID parameter)
>> {
>>   while (1)
>>    InterlockedIncrement(&value);
>>   return 0;
>> }
>> int __cdecl main()
>> {
>>   HANDLE thread =3D CreateThread(0, 0, Thread, 0, 0, 0);
>>   UINT i =3D 0;
>>   while (1)
>>   {
>>     i +=3D 1;
>>     if (SuspendThread(thread) =3D=3D (DWORD)-1)
>>     {
>>       printf("suspend failed %X\n", GetLastError())= >> ;
>>       Sleep(1);
>>       continue;
>>     }
>>     volatile long a =3D value;
>>     volatile long b =3D value;
>>     if (a !=3D b)
>>     {
>>      printf("%d %d %d %d\n", i, a, b, b - a);
>>     }
>>     ResumeThread(thread);
>>   }
>> }

>>
>>
>> >> >>
= >> >> --Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A-- From rodney_bates at lcwb.coop Mon Jan 20 17:00:13 2014 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 20 Jan 2014 10:00:13 -0600 Subject: [M3devel] narrow still failing.. In-Reply-To: <20140119220123.0D1971A208F@async.async.caltech.edu> References: <20140119213649.403AE1A208F@async.async.caltech.edu> <20140119220123.0D1971A208F@async.async.caltech.edu> Message-ID: <52DD480D.3000101@lcwb.coop> What does this do? PROCEDURE Z(VAR x : REFANY) = VAR b: BDD.T; BEGIN b := x; x := BDD.Format(b) END Z; The common property of the failing cases seems to be the necessity of a runtime narrow check when passing x to BDD.Format. So far, I haven't been able to reproduce the failure on AMD64_LINUX, with even more irrelevant (one would think) stuff pared out. I moved T, Root, and Format into main, removed all but one fields of T and Root, replaced BDD.New by NEW(Root) and removed Symtab and most of the body from Format. Looking at your code, I agree it should work. On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote: > As in the ISTYPE/TYPECASE example, the following works: > > > PROCEDURE Z(VAR x : REFANY) = > BEGIN > TYPECASE x OF > BDD.T(b) => x := BDD.Format(b) > END; > END Z; > > (244)rover:~/refany/src>../FreeBSD4/refany > a > > > This does NOT work: > > PROCEDURE Z(VAR x : REFANY) = > BEGIN > TYPECASE x OF > BDD.T(b) => x := BDD.Format(x) > END; > END Z; > > > (242)rover:~/refany/src>../FreeBSD4/refany > > > *** > *** runtime error: > *** NARROW failed > *** file "/big/home/mika/refany/src/Main.m3", line 22 > *** > > use option @M3stackdump to get a stack trace > Abort > > > Same result PM3 / CM3. > > Mika > From rodney_bates at lcwb.coop Mon Jan 20 18:02:40 2014 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 20 Jan 2014 11:02:40 -0600 Subject: [M3devel] narrow still failing.. In-Reply-To: <20140119220123.0D1971A208F@async.async.caltech.edu> References: <20140119213649.403AE1A208F@async.async.caltech.edu> <20140119220123.0D1971A208F@async.async.caltech.edu> Message-ID: <52DD56B0.1050007@lcwb.coop> Lots more experiments, still no reproduction of the problem on AMD64_LINUX. The 5 cases in the program below all work. But earlier versions of this test program, with bugs, (failure to repeat x:=y before every Z* call) suggested a hypothesis. For x := BDD.Format(x) Perhaps the compiler does something to the LHS x (zeros it out?) before narrowing and passing the x that is an actual parameter to Format. If so, it is an implementation bug, as the language says (2.3.1): (for v := e) "The order of evaluation of v and e is undefined, but e will be evaluated before v is updated." _____________________________________________________________________________________ MODULE Main; TYPE T = OBJECT l : T; END; TYPE Root = T OBJECT id : CARDINAL; END; PROCEDURE Format(x : T) : TEXT = BEGIN RETURN "Format" END Format; PROCEDURE P() : REFANY = BEGIN VAR x, y : REFANY; BEGIN y := NEW ( Root ); x := y; Z1(x); x := y; Z2(x); x := y; Z3(x); x := y; Z4(x); x := y; Z5(x); RETURN x END END P; PROCEDURE Z1(VAR x : REFANY) = BEGIN TYPECASE x OF T (b) => x := Format(b) END; END Z1; PROCEDURE Z2(VAR x : REFANY) = VAR b : T; BEGIN b := x; x := Format(b) END Z2; PROCEDURE Z3(VAR x : REFANY) = BEGIN x := Format(x) END Z3; PROCEDURE Z4(VAR x : REFANY) = BEGIN TYPECASE x OF T => x := Format (x) END; END Z4; PROCEDURE Z5(VAR x : REFANY) = BEGIN IF ISTYPE (x, T) THEN x := Format (x) END; END Z5; BEGIN EVAL P() END Main. ______________________________________________________________________________________ On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote: > As in the ISTYPE/TYPECASE example, the following works: > > > PROCEDURE Z(VAR x : REFANY) = > BEGIN > TYPECASE x OF > BDD.T(b) => x := BDD.Format(b) > END; > END Z; > > (244)rover:~/refany/src>../FreeBSD4/refany > a > > > This does NOT work: > > PROCEDURE Z(VAR x : REFANY) = > BEGIN > TYPECASE x OF > BDD.T(b) => x := BDD.Format(x) > END; > END Z; > > > (242)rover:~/refany/src>../FreeBSD4/refany > > > *** > *** runtime error: > *** NARROW failed > *** file "/big/home/mika/refany/src/Main.m3", line 22 > *** > > use option @M3stackdump to get a stack trace > Abort > > > Same result PM3 / CM3. > > Mika > From mika at async.caltech.edu Tue Jan 21 05:18:26 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Mon, 20 Jan 2014 20:18:26 -0800 Subject: [M3devel] narrow still failing.. In-Reply-To: <52DD480D.3000101@lcwb.coop> References: <20140119213649.403AE1A208F@async.async.caltech.edu> <20140119220123.0D1971A208F@async.async.caltech.edu> <52DD480D.3000101@lcwb.coop> Message-ID: <20140121041826.4D95D1A2080@async.async.caltech.edu> Were you able to get it to fail on any of your computers? Your Z fails for me---CM3 and PM3 both. (579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany *** *** runtime error: *** An explicit or implicit NARROW() operation failed. *** file "../src/Main.m3", line 30 *** Abort (580)truffles:~/bsd/refany/src>cat Mai Main.m3~ Main.m3 (580)truffles:~/bsd/refany/src>cat -n Main.m3 1 MODULE Main; 2 IMPORT BDD, IO; 3 4 PROCEDURE P() : REFANY = 5 BEGIN 6 VAR x : REFANY; 7 BEGIN 8 x := BDD.New("a"); 9 Z(x); 10 RETURN x 11 END 12 END P; 13 14 PROCEDURE Q() = 15 BEGIN 16 IO.Put(P() & "\n") 17 END Q; 18 19 (* 20 PROCEDURE Z(VAR x : REFANY) = 21 BEGIN 22 TYPECASE x OF 23 BDD.T(b) => x := BDD.Format(b) 24 END; 25 END Z; 26 *) 27 PROCEDURE Z(VAR x : REFANY) = 28 VAR b: BDD.T; 29 BEGIN 30 b := x; 31 x := BDD.Format(b) 32 END Z; 33 34 35 36 BEGIN 37 Q() 38 END Main. (581)truffles:~/bsd/refany/src> "Rodney M. Bates" writes: >What does this do? > >PROCEDURE Z(VAR x : REFANY) = > VAR b: BDD.T; > BEGIN > b := x; > x := BDD.Format(b) > END Z; > >The common property of the failing cases seems to be the necessity >of a runtime narrow check when passing x to BDD.Format. > >So far, I haven't been able to reproduce the failure on AMD64_LINUX, >with even more irrelevant (one would think) stuff pared out. I moved >T, Root, and Format into main, removed all but one fields of T and Root, >replaced BDD.New by NEW(Root) and removed Symtab and most of the body >from Format. > >Looking at your code, I agree it should work. > > >On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote: >> As in the ISTYPE/TYPECASE example, the following works: >> >> >> PROCEDURE Z(VAR x : REFANY) = >> BEGIN >> TYPECASE x OF >> BDD.T(b) => x := BDD.Format(b) >> END; >> END Z; >> >> (244)rover:~/refany/src>../FreeBSD4/refany >> a >> >> >> This does NOT work: >> >> PROCEDURE Z(VAR x : REFANY) = >> BEGIN >> TYPECASE x OF >> BDD.T(b) => x := BDD.Format(x) >> END; >> END Z; >> >> >> (242)rover:~/refany/src>../FreeBSD4/refany >> >> >> *** >> *** runtime error: >> *** NARROW failed >> *** file "/big/home/mika/refany/src/Main.m3", line 22 >> *** >> >> use option @M3stackdump to get a stack trace >> Abort >> >> >> Same result PM3 / CM3. >> >> Mika >> From rodney_bates at lcwb.coop Tue Jan 21 17:34:57 2014 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 21 Jan 2014 10:34:57 -0600 Subject: [M3devel] narrow still failing.. In-Reply-To: <20140121041826.4D95D1A2080@async.async.caltech.edu> References: <20140119213649.403AE1A208F@async.async.caltech.edu> <20140119220123.0D1971A208F@async.async.caltech.edu> <52DD480D.3000101@lcwb.coop> <20140121041826.4D95D1A2080@async.async.caltech.edu> Message-ID: <52DEA1B1.2070808@lcwb.coop> On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote: > Were you able to get it to fail on any of your computers? No, the only times I got narrow failures were due to errors in my trial cases. I have only tried AMD64_LINUX, but should be able to get this onto a LINUXLIBC6 system. > > Your Z fails for me---CM3 and PM3 both. > > (579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany > > > *** > *** runtime error: > *** An explicit or implicit NARROW() operation failed. > *** file "../src/Main.m3", line 30 > *** Well that torpedos both my theories. But it is hard to imagine that bad code generated for an implicit narrow on an assignment could ever have gone unnoticed so long. I thought it might be more believable if it only happened when narrowing an actual parameter before passing it. > > Abort > (580)truffles:~/bsd/refany/src>cat Mai > Main.m3~ Main.m3 > (580)truffles:~/bsd/refany/src>cat -n Main.m3 > 1 MODULE Main; > 2 IMPORT BDD, IO; > 3 > 4 PROCEDURE P() : REFANY = > 5 BEGIN > 6 VAR x : REFANY; > 7 BEGIN > 8 x := BDD.New("a"); > 9 Z(x); > 10 RETURN x > 11 END > 12 END P; > 13 > 14 PROCEDURE Q() = > 15 BEGIN > 16 IO.Put(P() & "\n") > 17 END Q; > 18 > 19 (* > 20 PROCEDURE Z(VAR x : REFANY) = > 21 BEGIN > 22 TYPECASE x OF > 23 BDD.T(b) => x := BDD.Format(b) > 24 END; > 25 END Z; > 26 *) > 27 PROCEDURE Z(VAR x : REFANY) = > 28 VAR b: BDD.T; > 29 BEGIN > 30 b := x; > 31 x := BDD.Format(b) > 32 END Z; > 33 > 34 > 35 > 36 BEGIN > 37 Q() > 38 END Main. > (581)truffles:~/bsd/refany/src> > "Rodney M. Bates" writes: >> What does this do? >> >> PROCEDURE Z(VAR x : REFANY) = >> VAR b: BDD.T; >> BEGIN >> b := x; >> x := BDD.Format(b) >> END Z; >> >> The common property of the failing cases seems to be the necessity >> of a runtime narrow check when passing x to BDD.Format. >> >> So far, I haven't been able to reproduce the failure on AMD64_LINUX, >> with even more irrelevant (one would think) stuff pared out. I moved >> T, Root, and Format into main, removed all but one fields of T and Root, >> replaced BDD.New by NEW(Root) and removed Symtab and most of the body >>from Format. >> >> Looking at your code, I agree it should work. >> >> >> On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote: >>> As in the ISTYPE/TYPECASE example, the following works: >>> >>> >>> PROCEDURE Z(VAR x : REFANY) = >>> BEGIN >>> TYPECASE x OF >>> BDD.T(b) => x := BDD.Format(b) >>> END; >>> END Z; >>> >>> (244)rover:~/refany/src>../FreeBSD4/refany >>> a >>> >>> >>> This does NOT work: >>> >>> PROCEDURE Z(VAR x : REFANY) = >>> BEGIN >>> TYPECASE x OF >>> BDD.T(b) => x := BDD.Format(x) >>> END; >>> END Z; >>> >>> >>> (242)rover:~/refany/src>../FreeBSD4/refany >>> >>> >>> *** >>> *** runtime error: >>> *** NARROW failed >>> *** file "/big/home/mika/refany/src/Main.m3", line 22 >>> *** >>> >>> use option @M3stackdump to get a stack trace >>> Abort >>> >>> >>> Same result PM3 / CM3. >>> >>> Mika >>> > From mika at async.caltech.edu Tue Jan 21 18:11:20 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Tue, 21 Jan 2014 09:11:20 -0800 Subject: [M3devel] narrow still failing.. In-Reply-To: <52DEA1B1.2070808@lcwb.coop> References: <20140119213649.403AE1A208F@async.async.caltech.edu> <20140119220123.0D1971A208F@async.async.caltech.edu> <52DD480D.3000101@lcwb.coop> <20140121041826.4D95D1A2080@async.async.caltech.edu> <52DEA1B1.2070808@lcwb.coop> Message-ID: <20140121171120.F21FF1A2080@async.async.caltech.edu> Did you try using my bdd.tgz ? I am getting this error on: ancient PM3 on FreeBSD4 head CM3 on AMD64_LINUX few months old CM3 on ARM_LINUX (Raspberry Pi) I doubt you will get it on LINUXLIBC6 if you don't get it on the others. I already tried rearranging the files (the implementation and interfaces are split in a funny way across BDD.i3, BDDImpl.i3, BDD.m3, BDDImpl.m3 right now). With no effect. I don't know what it is about my code that's tripping up the compiler. It's also a "decent" BDD package so someone (me I guess, but anyone else who's interested is welcome to) might want to add it to CM3... :-) Kind of a pain if you can't print out the debugging info though! (Actually the workarounds are OK.) Mika "Rodney M. Bates" writes: > > >On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote: >> Were you able to get it to fail on any of your computers? > >No, the only times I got narrow failures were due to errors >in my trial cases. > >I have only tried AMD64_LINUX, but should be able to get this onto a >LINUXLIBC6 system. > >> >> Your Z fails for me---CM3 and PM3 both. >> >> (579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany >> >> >> *** >> *** runtime error: >> *** An explicit or implicit NARROW() operation failed. >> *** file "../src/Main.m3", line 30 >> *** > >Well that torpedos both my theories. > >But it is hard to imagine that bad code generated for an implicit narrow >on an assignment could ever have gone unnoticed so long. I thought it >might be more believable if it only happened when narrowing an actual >parameter before passing it. > >> >> Abort >> (580)truffles:~/bsd/refany/src>cat Mai >> Main.m3~ Main.m3 >> (580)truffles:~/bsd/refany/src>cat -n Main.m3 >> 1 MODULE Main; >> 2 IMPORT BDD, IO; >> 3 >> 4 PROCEDURE P() : REFANY = >> 5 BEGIN >> 6 VAR x : REFANY; >> 7 BEGIN >> 8 x := BDD.New("a"); >> 9 Z(x); >> 10 RETURN x >> 11 END >> 12 END P; >> 13 >> 14 PROCEDURE Q() = >> 15 BEGIN >> 16 IO.Put(P() & "\n") >> 17 END Q; >> 18 >> 19 (* >> 20 PROCEDURE Z(VAR x : REFANY) = >> 21 BEGIN >> 22 TYPECASE x OF >> 23 BDD.T(b) => x := BDD.Format(b) >> 24 END; >> 25 END Z; >> 26 *) >> 27 PROCEDURE Z(VAR x : REFANY) = >> 28 VAR b: BDD.T; >> 29 BEGIN >> 30 b := x; >> 31 x := BDD.Format(b) >> 32 END Z; >> 33 >> 34 >> 35 >> 36 BEGIN >> 37 Q() >> 38 END Main. >> (581)truffles:~/bsd/refany/src> >> "Rodney M. Bates" writes: >>> What does this do? >>> >>> PROCEDURE Z(VAR x : REFANY) = >>> VAR b: BDD.T; >>> BEGIN >>> b := x; >>> x := BDD.Format(b) >>> END Z; >>> >>> The common property of the failing cases seems to be the necessity >>> of a runtime narrow check when passing x to BDD.Format. >>> >>> So far, I haven't been able to reproduce the failure on AMD64_LINUX, >>> with even more irrelevant (one would think) stuff pared out. I moved >>> T, Root, and Format into main, removed all but one fields of T and Root, >>> replaced BDD.New by NEW(Root) and removed Symtab and most of the body >>>from Format. >>> >>> Looking at your code, I agree it should work. >>> >>> >>> On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote: >>>> As in the ISTYPE/TYPECASE example, the following works: >>>> >>>> >>>> PROCEDURE Z(VAR x : REFANY) = >>>> BEGIN >>>> TYPECASE x OF >>>> BDD.T(b) => x := BDD.Format(b) >>>> END; >>>> END Z; >>>> >>>> (244)rover:~/refany/src>../FreeBSD4/refany >>>> a >>>> >>>> >>>> This does NOT work: >>>> >>>> PROCEDURE Z(VAR x : REFANY) = >>>> BEGIN >>>> TYPECASE x OF >>>> BDD.T(b) => x := BDD.Format(x) >>>> END; >>>> END Z; >>>> >>>> >>>> (242)rover:~/refany/src>../FreeBSD4/refany >>>> >>>> >>>> *** >>>> *** runtime error: >>>> *** NARROW failed >>>> *** file "/big/home/mika/refany/src/Main.m3", line 22 >>>> *** >>>> >>>> use option @M3stackdump to get a stack trace >>>> Abort >>>> >>>> >>>> Same result PM3 / CM3. >>>> >>>> Mika >>>> >> From rodney_bates at lcwb.coop Tue Jan 21 18:23:41 2014 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 21 Jan 2014 11:23:41 -0600 Subject: [M3devel] narrow still failing.. In-Reply-To: <20140121041826.4D95D1A2080@async.async.caltech.edu> References: <20140119213649.403AE1A208F@async.async.caltech.edu> <20140119220123.0D1971A208F@async.async.caltech.edu> <52DD480D.3000101@lcwb.coop> <20140121041826.4D95D1A2080@async.async.caltech.edu> Message-ID: <52DEAD1D.8010602@lcwb.coop> On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote: > Were you able to get it to fail on any of your computers? > My test case with 5 versions of Z, and your complete one that you posted (bdd.tgz) both silently complete with no error messages on LINUXLIBC6, using the head compiler and libraries. bdd.tgz also works fine on AMD64_LINUX. On yours, I built and shipped in bdd/src, then built in bdd/test/src and ran LINUXLIBC6/bddtest, which I presume was the right procedure. How about compiling a couple of failing Z versions on a machine where they fail, with -keep, then posting the .mc, .ms, and .mo files and corresponding source (only for the Zs is probably enough). >>> Mika >>> > From rodney_bates at lcwb.coop Tue Jan 21 19:02:09 2014 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 21 Jan 2014 12:02:09 -0600 Subject: [M3devel] narrow still failing.. In-Reply-To: <20140121171120.F21FF1A2080@async.async.caltech.edu> References: <20140119213649.403AE1A208F@async.async.caltech.edu> <20140119220123.0D1971A208F@async.async.caltech.edu> <52DD480D.3000101@lcwb.coop> <20140121041826.4D95D1A2080@async.async.caltech.edu> <52DEA1B1.2070808@lcwb.coop> <20140121171120.F21FF1A2080@async.async.caltech.edu> Message-ID: <52DEB621.7080706@lcwb.coop> On 01/21/2014 11:11 AM, mika at async.caltech.edu wrote: > Did you try using my bdd.tgz ? > > I am getting this error on: > > ancient PM3 on FreeBSD4 > > head CM3 on AMD64_LINUX Hmm. I can't get any errors compiling bdd and bddtest on my AMD64_LINUX system with a freshly updated head. Hacking around, I tried to use the binaries in bdd.tgz, but my .M3SHIP wants a libbdd.so.5, not there. Some of the *.[im]o files are slightly different in size when I recompiled, but I think there is an explanation for that. I can't build bddtest with the release compiler, because it needs caltech_parser, and that won't build in the release. This combination of [non]failures seems pretty strange. I am also wondering about a problem with type checks in the runtime. > > few months old CM3 on ARM_LINUX (Raspberry Pi) > > I doubt you will get it on LINUXLIBC6 if you don't get it on the others. > > I already tried rearranging the files (the implementation and interfaces > are split in a funny way across BDD.i3, BDDImpl.i3, BDD.m3, BDDImpl.m3 > right now). With no effect. I don't know what it is about my code that's > tripping up the compiler. > > It's also a "decent" BDD package so someone (me I guess, but anyone > else who's interested is welcome to) might want to add it to CM3... :-) > Kind of a pain if you can't print out the debugging info though! > (Actually the workarounds are OK.) > > Mika > > "Rodney M. Bates" writes: >> >> >> On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote: >>> Were you able to get it to fail on any of your computers? >> >> No, the only times I got narrow failures were due to errors >> in my trial cases. >> >> I have only tried AMD64_LINUX, but should be able to get this onto a >> LINUXLIBC6 system. >> >>> >>> Your Z fails for me---CM3 and PM3 both. >>> >>> (579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany >>> >>> >>> *** >>> *** runtime error: >>> *** An explicit or implicit NARROW() operation failed. >>> *** file "../src/Main.m3", line 30 >>> *** >> >> Well that torpedos both my theories. >> >> But it is hard to imagine that bad code generated for an implicit narrow >> on an assignment could ever have gone unnoticed so long. I thought it >> might be more believable if it only happened when narrowing an actual >> parameter before passing it. >> >>> >>> Abort >>> (580)truffles:~/bsd/refany/src>cat Mai >>> Main.m3~ Main.m3 >>> (580)truffles:~/bsd/refany/src>cat -n Main.m3 >>> 1 MODULE Main; >>> 2 IMPORT BDD, IO; >>> 3 >>> 4 PROCEDURE P() : REFANY = >>> 5 BEGIN >>> 6 VAR x : REFANY; >>> 7 BEGIN >>> 8 x := BDD.New("a"); >>> 9 Z(x); >>> 10 RETURN x >>> 11 END >>> 12 END P; >>> 13 >>> 14 PROCEDURE Q() = >>> 15 BEGIN >>> 16 IO.Put(P() & "\n") >>> 17 END Q; >>> 18 >>> 19 (* >>> 20 PROCEDURE Z(VAR x : REFANY) = >>> 21 BEGIN >>> 22 TYPECASE x OF >>> 23 BDD.T(b) => x := BDD.Format(b) >>> 24 END; >>> 25 END Z; >>> 26 *) >>> 27 PROCEDURE Z(VAR x : REFANY) = >>> 28 VAR b: BDD.T; >>> 29 BEGIN >>> 30 b := x; >>> 31 x := BDD.Format(b) >>> 32 END Z; >>> 33 >>> 34 >>> 35 >>> 36 BEGIN >>> 37 Q() >>> 38 END Main. >>> (581)truffles:~/bsd/refany/src> >>> "Rodney M. Bates" writes: >>>> What does this do? >>>> >>>> PROCEDURE Z(VAR x : REFANY) = >>>> VAR b: BDD.T; >>>> BEGIN >>>> b := x; >>>> x := BDD.Format(b) >>>> END Z; >>>> >>>> The common property of the failing cases seems to be the necessity >>>> of a runtime narrow check when passing x to BDD.Format. >>>> >>>> So far, I haven't been able to reproduce the failure on AMD64_LINUX, >>>> with even more irrelevant (one would think) stuff pared out. I moved >>>> T, Root, and Format into main, removed all but one fields of T and Root, >>>> replaced BDD.New by NEW(Root) and removed Symtab and most of the body >>> >from Format. >>>> >>>> Looking at your code, I agree it should work. >>>> >>>> >>>> On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote: >>>>> As in the ISTYPE/TYPECASE example, the following works: >>>>> >>>>> >>>>> PROCEDURE Z(VAR x : REFANY) = >>>>> BEGIN >>>>> TYPECASE x OF >>>>> BDD.T(b) => x := BDD.Format(b) >>>>> END; >>>>> END Z; >>>>> >>>>> (244)rover:~/refany/src>../FreeBSD4/refany >>>>> a >>>>> >>>>> >>>>> This does NOT work: >>>>> >>>>> PROCEDURE Z(VAR x : REFANY) = >>>>> BEGIN >>>>> TYPECASE x OF >>>>> BDD.T(b) => x := BDD.Format(x) >>>>> END; >>>>> END Z; >>>>> >>>>> >>>>> (242)rover:~/refany/src>../FreeBSD4/refany >>>>> >>>>> >>>>> *** >>>>> *** runtime error: >>>>> *** NARROW failed >>>>> *** file "/big/home/mika/refany/src/Main.m3", line 22 >>>>> *** >>>>> >>>>> use option @M3stackdump to get a stack trace >>>>> Abort >>>>> >>>>> >>>>> Same result PM3 / CM3. >>>>> >>>>> Mika >>>>> >>> > From mika at async.caltech.edu Tue Jan 21 19:08:06 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Tue, 21 Jan 2014 10:08:06 -0800 Subject: [M3devel] narrow still failing.. In-Reply-To: <52DEB621.7080706@lcwb.coop> References: <20140119213649.403AE1A208F@async.async.caltech.edu> <20140119220123.0D1971A208F@async.async.caltech.edu> <52DD480D.3000101@lcwb.coop> <20140121041826.4D95D1A2080@async.async.caltech.edu> <52DEA1B1.2070808@lcwb.coop> <20140121171120.F21FF1A2080@async.async.caltech.edu> <52DEB621.7080706@lcwb.coop> Message-ID: <20140121180806.D75A61A208F@async.async.caltech.edu> The only thing bddtest uses from caltech_parser is Debug.Out. You can just take out those calls or replace them with IO.Put. But again bddtest is not the program that fails... it's the "refany" program (now in a sibling directory to bddtest) Mika "Rodney M. Bates" writes: > > >On 01/21/2014 11:11 AM, mika at async.caltech.edu wrote: >> Did you try using my bdd.tgz ? >> >> I am getting this error on: >> >> ancient PM3 on FreeBSD4 >> >> head CM3 on AMD64_LINUX > >Hmm. I can't get any errors compiling bdd and bddtest on my AMD64_LINUX system with >a freshly updated head. Hacking around, I tried to use the binaries in bdd.tgz, >but my .M3SHIP wants a libbdd.so.5, not there. Some of the *.[im]o files are >slightly different in size when I recompiled, but I think there is an explanation >for that. > >I can't build bddtest with the release compiler, because it needs caltech_parser, >and that won't build in the release. This combination of [non]failures seems >pretty strange. I am also wondering about a problem with type checks in the >runtime. > > >> >> few months old CM3 on ARM_LINUX (Raspberry Pi) >> >> I doubt you will get it on LINUXLIBC6 if you don't get it on the others. >> >> I already tried rearranging the files (the implementation and interfaces >> are split in a funny way across BDD.i3, BDDImpl.i3, BDD.m3, BDDImpl.m3 >> right now). With no effect. I don't know what it is about my code that's >> tripping up the compiler. >> >> It's also a "decent" BDD package so someone (me I guess, but anyone >> else who's interested is welcome to) might want to add it to CM3... :-) >> Kind of a pain if you can't print out the debugging info though! >> (Actually the workarounds are OK.) >> >> Mika >> >> "Rodney M. Bates" writes: >>> >>> >>> On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote: >>>> Were you able to get it to fail on any of your computers? >>> >>> No, the only times I got narrow failures were due to errors >>> in my trial cases. >>> >>> I have only tried AMD64_LINUX, but should be able to get this onto a >>> LINUXLIBC6 system. >>> >>>> >>>> Your Z fails for me---CM3 and PM3 both. >>>> >>>> (579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany >>>> >>>> >>>> *** >>>> *** runtime error: >>>> *** An explicit or implicit NARROW() operation failed. >>>> *** file "../src/Main.m3", line 30 >>>> *** >>> >>> Well that torpedos both my theories. >>> >>> But it is hard to imagine that bad code generated for an implicit narrow >>> on an assignment could ever have gone unnoticed so long. I thought it >>> might be more believable if it only happened when narrowing an actual >>> parameter before passing it. >>> >>>> >>>> Abort >>>> (580)truffles:~/bsd/refany/src>cat Mai >>>> Main.m3~ Main.m3 >>>> (580)truffles:~/bsd/refany/src>cat -n Main.m3 >>>> 1 MODULE Main; >>>> 2 IMPORT BDD, IO; >>>> 3 >>>> 4 PROCEDURE P() : REFANY = >>>> 5 BEGIN >>>> 6 VAR x : REFANY; >>>> 7 BEGIN >>>> 8 x := BDD.New("a"); >>>> 9 Z(x); >>>> 10 RETURN x >>>> 11 END >>>> 12 END P; >>>> 13 >>>> 14 PROCEDURE Q() = >>>> 15 BEGIN >>>> 16 IO.Put(P() & "\n") >>>> 17 END Q; >>>> 18 >>>> 19 (* >>>> 20 PROCEDURE Z(VAR x : REFANY) = >>>> 21 BEGIN >>>> 22 TYPECASE x OF >>>> 23 BDD.T(b) => x := BDD.Format(b) >>>> 24 END; >>>> 25 END Z; >>>> 26 *) >>>> 27 PROCEDURE Z(VAR x : REFANY) = >>>> 28 VAR b: BDD.T; >>>> 29 BEGIN >>>> 30 b := x; >>>> 31 x := BDD.Format(b) >>>> 32 END Z; >>>> 33 >>>> 34 >>>> 35 >>>> 36 BEGIN >>>> 37 Q() >>>> 38 END Main. >>>> (581)truffles:~/bsd/refany/src> >>>> "Rodney M. Bates" writes: >>>>> What does this do? >>>>> >>>>> PROCEDURE Z(VAR x : REFANY) = >>>>> VAR b: BDD.T; >>>>> BEGIN >>>>> b := x; >>>>> x := BDD.Format(b) >>>>> END Z; >>>>> >>>>> The common property of the failing cases seems to be the necessity >>>>> of a runtime narrow check when passing x to BDD.Format. >>>>> >>>>> So far, I haven't been able to reproduce the failure on AMD64_LINUX, >>>>> with even more irrelevant (one would think) stuff pared out. I moved >>>>> T, Root, and Format into main, removed all but one fields of T and Root, >>>>> replaced BDD.New by NEW(Root) and removed Symtab and most of the body >>>> >from Format. >>>>> >>>>> Looking at your code, I agree it should work. >>>>> >>>>> >>>>> On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote: >>>>>> As in the ISTYPE/TYPECASE example, the following works: >>>>>> >>>>>> >>>>>> PROCEDURE Z(VAR x : REFANY) = >>>>>> BEGIN >>>>>> TYPECASE x OF >>>>>> BDD.T(b) => x := BDD.Format(b) >>>>>> END; >>>>>> END Z; >>>>>> >>>>>> (244)rover:~/refany/src>../FreeBSD4/refany >>>>>> a >>>>>> >>>>>> >>>>>> This does NOT work: >>>>>> >>>>>> PROCEDURE Z(VAR x : REFANY) = >>>>>> BEGIN >>>>>> TYPECASE x OF >>>>>> BDD.T(b) => x := BDD.Format(x) >>>>>> END; >>>>>> END Z; >>>>>> >>>>>> >>>>>> (242)rover:~/refany/src>../FreeBSD4/refany >>>>>> >>>>>> >>>>>> *** >>>>>> *** runtime error: >>>>>> *** NARROW failed >>>>>> *** file "/big/home/mika/refany/src/Main.m3", line 22 >>>>>> *** >>>>>> >>>>>> use option @M3stackdump to get a stack trace >>>>>> Abort >>>>>> >>>>>> >>>>>> Same result PM3 / CM3. >>>>>> >>>>>> Mika >>>>>> >>>> >> From mika at async.caltech.edu Tue Jan 21 20:23:28 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Tue, 21 Jan 2014 11:23:28 -0800 Subject: [M3devel] narrow still failing.. In-Reply-To: <07CB74B6-895A-4053-9999-1D1A46C404D0@purdue.edu> References: <20140119213649.403AE1A208F@async.async.caltech.edu> <20140119220123.0D1971A208F@async.async.caltech.edu> <52DD480D.3000101@lcwb.coop> <20140121041826.4D95D1A2080@async.async.caltech.edu> <52DEA1B1.2070808@lcwb.coop> <20140121171120.F21FF1A2080@async.async.caltech.edu> <52DEB621.7080706@lcwb.coop> <20140121180806.D75A61A208F@async.async.caltech.edu> <07CB74B6-895A-4053-9999-1D1A46C404D0@purdue.edu> Message-ID: <20140121192328.92DDB1A208F@async.async.caltech.edu> How do you get them into text form? They look binary... I copied the whole directory bdd (and all subdirs) to http://rover.gcapltd.com/~mika/bdd/ Antony Hosking writes: >Please post .io / .mo in text form. > >Sent from my iPad > >> On Jan 21, 2014, at 1:08 PM, mika at async.caltech.edu wrote: >>=20 >> The only thing bddtest uses from caltech_parser is Debug.Out. You can jus= >t take out those calls or replace them with IO.Put. >>=20 >> But again bddtest is not the program that fails... it's the "refany" progr= >am (now in a sibling directory to bddtest) >>=20 >> Mika >>=20 >> "Rodney M. Bates" writes: >>>=20 >>>=20 >>>> On 01/21/2014 11:11 AM, mika at async.caltech.edu wrote: >>>> Did you try using my bdd.tgz ? >>>>=20 >>>> I am getting this error on: >>>>=20 >>>> ancient PM3 on FreeBSD4 >>>>=20 >>>> head CM3 on AMD64_LINUX >>>=20 >>> Hmm. I can't get any errors compiling bdd and bddtest on my AMD64_LINUX s= >ystem with >>> a freshly updated head. Hacking around, I tried to use the binaries in b= >dd.tgz, >>> but my .M3SHIP wants a libbdd.so.5, not there. Some of the *.[im]o files= > are >>> slightly different in size when I recompiled, but I think there is an exp= >lanation >>> for that. >>>=20 >>> I can't build bddtest with the release compiler, because it needs caltech= >_parser, >>> and that won't build in the release. This combination of [non]failures s= >eems >>> pretty strange. I am also wondering about a problem with type checks in t= >he >>> runtime. >>>=20 >>>=20 >>>>=20 >>>> few months old CM3 on ARM_LINUX (Raspberry Pi) >>>>=20 >>>> I doubt you will get it on LINUXLIBC6 if you don't get it on the others.= > >>>>=20 >>>> I already tried rearranging the files (the implementation and interfaces= > >>>> are split in a funny way across BDD.i3, BDDImpl.i3, BDD.m3, BDDImpl.m3 >>>> right now). With no effect. I don't know what it is about my code that= >'s >>>> tripping up the compiler. >>>>=20 >>>> It's also a "decent" BDD package so someone (me I guess, but anyone >>>> else who's interested is welcome to) might want to add it to CM3... :-) >>>> Kind of a pain if you can't print out the debugging info though! >>>> (Actually the workarounds are OK.) >>>>=20 >>>> Mika >>>>=20 >>>> "Rodney M. Bates" writes: >>>>>=20 >>>>>=20 >>>>>> On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote: >>>>>> Were you able to get it to fail on any of your computers? >>>>>=20 >>>>> No, the only times I got narrow failures were due to errors >>>>> in my trial cases. >>>>>=20 >>>>> I have only tried AMD64_LINUX, but should be able to get this onto a >>>>> LINUXLIBC6 system. >>>>>=20 >>>>>>=20 >>>>>> Your Z fails for me---CM3 and PM3 both. >>>>>>=20 >>>>>> (579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany >>>>>>=20 >>>>>>=20 >>>>>> *** >>>>>> *** runtime error: >>>>>> *** An explicit or implicit NARROW() operation failed. >>>>>> *** file "../src/Main.m3", line 30 >>>>>> *** >>>>>=20 >>>>> Well that torpedos both my theories. >>>>>=20 >>>>> But it is hard to imagine that bad code generated for an implicit narro= >w >>>>> on an assignment could ever have gone unnoticed so long. I thought it >>>>> might be more believable if it only happened when narrowing an actual >>>>> parameter before passing it. >>>>>=20 >>>>>>=20 >>>>>> Abort >>>>>> (580)truffles:~/bsd/refany/src>cat Mai >>>>>> Main.m3~ Main.m3 >>>>>> (580)truffles:~/bsd/refany/src>cat -n Main.m3 >>>>>> 1 MODULE Main; >>>>>> 2 IMPORT BDD, IO; >>>>>> 3 >>>>>> 4 PROCEDURE P() : REFANY =3D >>>>>> 5 BEGIN >>>>>> 6 VAR x : REFANY; >>>>>> 7 BEGIN >>>>>> 8 x :=3D BDD.New("a"); >>>>>> 9 Z(x); >>>>>> 10 RETURN x >>>>>> 11 END >>>>>> 12 END P; >>>>>> 13 >>>>>> 14 PROCEDURE Q() =3D >>>>>> 15 BEGIN >>>>>> 16 IO.Put(P() & "\n") >>>>>> 17 END Q; >>>>>> 18 >>>>>> 19 (* >>>>>> 20 PROCEDURE Z(VAR x : REFANY) =3D >>>>>> 21 BEGIN >>>>>> 22 TYPECASE x OF >>>>>> 23 BDD.T(b) =3D> x :=3D BDD.Format(b) >>>>>> 24 END; >>>>>> 25 END Z; >>>>>> 26 *) >>>>>> 27 PROCEDURE Z(VAR x : REFANY) =3D >>>>>> 28 VAR b: BDD.T; >>>>>> 29 BEGIN >>>>>> 30 b :=3D x; >>>>>> 31 x :=3D BDD.Format(b) >>>>>> 32 END Z; >>>>>> 33 >>>>>> 34 >>>>>> 35 >>>>>> 36 BEGIN >>>>>> 37 Q() >>>>>> 38 END Main. >>>>>> (581)truffles:~/bsd/refany/src> >>>>>> "Rodney M. Bates" writes: >>>>>>> What does this do? >>>>>>>=20 >>>>>>> PROCEDURE Z(VAR x : REFANY) =3D >>>>>>> VAR b: BDD.T; >>>>>>> BEGIN >>>>>>> b :=3D x; >>>>>>> x :=3D BDD.Format(b) >>>>>>> END Z; >>>>>>>=20 >>>>>>> The common property of the failing cases seems to be the necessity >>>>>>> of a runtime narrow check when passing x to BDD.Format. >>>>>>>=20 >>>>>>> So far, I haven't been able to reproduce the failure on AMD64_LINUX, >>>>>>> with even more irrelevant (one would think) stuff pared out. I moved= > >>>>>>> T, Root, and Format into main, removed all but one fields of T and Ro= >ot, >>>>>>> replaced BDD.New by NEW(Root) and removed Symtab and most of the body= > >>>>>>> from Format. >>>>>>>=20 >>>>>>> Looking at your code, I agree it should work. >>>>>>>=20 >>>>>>>=20 >>>>>>>> On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote: >>>>>>>> As in the ISTYPE/TYPECASE example, the following works: >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> PROCEDURE Z(VAR x : REFANY) =3D >>>>>>>> BEGIN >>>>>>>> TYPECASE x OF >>>>>>>> BDD.T(b) =3D> x :=3D BDD.Format(b) >>>>>>>> END; >>>>>>>> END Z; >>>>>>>>=20 >>>>>>>> (244)rover:~/refany/src>../FreeBSD4/refany >>>>>>>> a >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> This does NOT work: >>>>>>>>=20 >>>>>>>> PROCEDURE Z(VAR x : REFANY) =3D >>>>>>>> BEGIN >>>>>>>> TYPECASE x OF >>>>>>>> BDD.T(b) =3D> x :=3D BDD.Format(x) >>>>>>>> END; >>>>>>>> END Z; >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> (242)rover:~/refany/src>../FreeBSD4/refany >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> *** >>>>>>>> *** runtime error: >>>>>>>> *** NARROW failed >>>>>>>> *** file "/big/home/mika/refany/src/Main.m3", line 22 >>>>>>>> *** >>>>>>>>=20 >>>>>>>> use option @M3stackdump to get a stack trace >>>>>>>> Abort >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> Same result PM3 / CM3. >>>>>>>>=20 >>>>>>>> Mika >>>>=20 From rodney_bates at lcwb.coop Wed Jan 22 20:46:55 2014 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 22 Jan 2014 13:46:55 -0600 Subject: [M3devel] narrow still failing.. In-Reply-To: <20140119220123.0D1971A208F@async.async.caltech.edu> References: <20140119213649.403AE1A208F@async.async.caltech.edu> <20140119220123.0D1971A208F@async.async.caltech.edu> Message-ID: <52E0202F.3020307@lcwb.coop> Some more info: The front end is generating incorrect narrow code, only where the type T is an unrevealed <: REFANY. When T and Root are known, it calls RTHooks.CheckIsType for ISTYPE, NARROW, and implicit narrow in an assignment. When opaque, for explicit and implicit narrow, it simply compares the typecode of x to a single typecode (no doubt for T) and demands they be equal. In this case, the actual allocated type of x is Root, a proper subtype of T. For ISTYPE, it does call CheckIsType even when the types are opaque. The fact that it happens only when the types are opaque lends credibility to the fact that this bug has gone unnoticed at least as far back as PM3. On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote: > As in the ISTYPE/TYPECASE example, the following works: > > > PROCEDURE Z(VAR x : REFANY) = > BEGIN > TYPECASE x OF > BDD.T(b) => x := BDD.Format(b) > END; > END Z; > > (244)rover:~/refany/src>../FreeBSD4/refany > a > > > This does NOT work: > > PROCEDURE Z(VAR x : REFANY) = > BEGIN > TYPECASE x OF > BDD.T(b) => x := BDD.Format(x) > END; > END Z; > > > (242)rover:~/refany/src>../FreeBSD4/refany > > > *** > *** runtime error: > *** NARROW failed > *** file "/big/home/mika/refany/src/Main.m3", line 22 > *** > > use option @M3stackdump to get a stack trace > Abort > > > Same result PM3 / CM3. > > Mika > From mika at async.caltech.edu Wed Jan 22 21:47:49 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Wed, 22 Jan 2014 12:47:49 -0800 Subject: [M3devel] narrow still failing.. In-Reply-To: <52E0202F.3020307@lcwb.coop> References: <20140119213649.403AE1A208F@async.async.caltech.edu> <20140119220123.0D1971A208F@async.async.caltech.edu> <52E0202F.3020307@lcwb.coop> Message-ID: <20140122204749.C724E1A208F@async.async.caltech.edu> Aha so declaring the type as <: ROOT will probably work as a workaround? Thank you for looking into this. "Rodney M. Bates" writes: >Some more info: > >The front end is generating incorrect narrow code, only >where the type T is an unrevealed <: REFANY. When T and Root >are known, it calls RTHooks.CheckIsType for ISTYPE, NARROW, >and implicit narrow in an assignment. > >When opaque, for explicit and implicit narrow, it simply compares >the typecode of x to a single typecode (no doubt for T) and demands >they be equal. In this case, the actual allocated type of x is Root, >a proper subtype of T. > >For ISTYPE, it does call CheckIsType even when the types are opaque. > >The fact that it happens only when the types are opaque lends >credibility to the fact that this bug has gone unnoticed at least >as far back as PM3. From jay.krell at cornell.edu Thu Jan 23 08:51:58 2014 From: jay.krell at cornell.edu (Jay K) Date: Thu, 23 Jan 2014 07:51:58 +0000 Subject: [M3devel] proposal for getting wow64 context/stack reliably? Message-ID: There is a behavior/bug in wow64. Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc. SuspendThread / GetThreadContext work like this: 32bit processes consist almost entirely of 32bit code. There is a small amount of 64bit code. If you suspend while running 32bit code, GetThreadContext works. if you suspend while running 64bit code, GetThreadContext usually but not always works. 64bit code is run en route to syscalls. For example you call: 1 kernel32!Sleep 2 it calls 32bit NtDelayExecution 3 that calls wow64NtDelayExecutation (via a cross segment "far" jmp or call) 4 which calls native NtDelayExecuation In between 2 and 3, within 64bit code, the 32bit context is saved. You can step through it very easily in a debugger. Really. Where GetThreadContext knows where to get it. The problem is that saving context is not atomic. You can suspend while saving context. What to do? scratch/wow64stack contains a program that detects the bug. I believe it is the basis of a workaround for the bug. Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, not only functions that use exception handling, but also functions that call "extern" functions call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection use that, if it isn't zero. Normally it will be zero. Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile. That is why it isn't a set/set-zero pattern (like in the test program). Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. Rejected counter proposal: Don't suspend/gc when running a syscall. No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that. Possible augmentation: if running native, short circuit most of this Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, until/unless there is another 32bit platform runable on some similar 64bit platforms. Performance impact: hypothetically large but probably not noticable. Furthe refinements: It isn't extern/native code per se, it is syscalls. We could further augment pragmas to discern them. We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... Agreed? I'll make the compiler change? Oh, also, not just stack pointer, but other registers, at least non-volatiles? Eventually cooperative suspend will cause this to fall away as a problem.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcolebur at SCIRES.COM Thu Jan 23 12:59:34 2014 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Thu, 23 Jan 2014 11:59:34 +0000 Subject: [M3devel] EXT: proposal for getting wow64 context/stack reliably? In-Reply-To: References: Message-ID: Jay, If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows? --Randy Sent from my iPhone On Jan 23, 2014, at 2:58 AM, "Jay K" > wrote: There is a behavior/bug in wow64. Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc. SuspendThread / GetThreadContext work like this: 32bit processes consist almost entirely of 32bit code. There is a small amount of 64bit code. If you suspend while running 32bit code, GetThreadContext works. if you suspend while running 64bit code, GetThreadContext usually but not always works. 64bit code is run en route to syscalls. For example you call: 1 kernel32!Sleep 2 it calls 32bit NtDelayExecution 3 that calls wow64NtDelayExecutation (via a cross segment "far" jmp or call) 4 which calls native NtDelayExecuation In between 2 and 3, within 64bit code, the 32bit context is saved. You can step through it very easily in a debugger. Really. Where GetThreadContext knows where to get it. The problem is that saving context is not atomic. You can suspend while saving context. What to do? scratch/wow64stack contains a program that detects the bug. I believe it is the basis of a workaround for the bug. Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, not only functions that use exception handling, but also functions that call "extern" functions call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection use that, if it isn't zero. Normally it will be zero. Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile. That is why it isn't a set/set-zero pattern (like in the test program). Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. Rejected counter proposal: Don't suspend/gc when running a syscall. No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that. Possible augmentation: if running native, short circuit most of this Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, until/unless there is another 32bit platform runable on some similar 64bit platforms. Performance impact: hypothetically large but probably not noticable. Furthe refinements: It isn't extern/native code per se, it is syscalls. We could further augment pragmas to discern them. We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... Agreed? I'll make the compiler change? Oh, also, not just stack pointer, but other registers, at least non-volatiles? Eventually cooperative suspend will cause this to fall away as a problem.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Thu Jan 23 16:25:59 2014 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 23 Jan 2014 09:25:59 -0600 Subject: [M3devel] narrow still failing.. In-Reply-To: <20140122204749.C724E1A208F@async.async.caltech.edu> References: <20140119213649.403AE1A208F@async.async.caltech.edu> <20140119220123.0D1971A208F@async.async.caltech.edu> <52E0202F.3020307@lcwb.coop> <20140122204749.C724E1A208F@async.async.caltech.edu> Message-ID: <52E13487.9090504@lcwb.coop> On 01/22/2014 02:47 PM, mika at async.caltech.edu wrote: > Aha so declaring the type as <: ROOT will probably work as a workaround? > I'm not sure whether that would do it or not. In my pared-down case, I had moved BDD.T and BDD.Root into main, but left the REFANY variables. That made the failures go away. Until it is fixed, I think the best workaround is TYPECASE with a binding: TYPECASE x OF BDD.T(b) => Format(b), etc. This is known to be working now. You've probably thought about this, but this way requires only a single runtime type test. Either TYPECASE without the binding of b, or ISTYPE, will require an additional recheck of the same thing when the narrow, implicit or explicit, happens. I doubt even a pretty good optimizing compiler could remove this. For ISTYPE, it would have to know that the runtime routine (e.g. CheckIsType) was a pure function. Pretty unlikely. Even worse for TYPECASE, where it would have to know a lot about the relation between CheckIsType and ScanTypecase. > Thank you for looking into this. > > "Rodney M. Bates" writes: >> Some more info: >> >> The front end is generating incorrect narrow code, only >> where the type T is an unrevealed <: REFANY. When T and Root >> are known, it calls RTHooks.CheckIsType for ISTYPE, NARROW, >> and implicit narrow in an assignment. >> >> When opaque, for explicit and implicit narrow, it simply compares >> the typecode of x to a single typecode (no doubt for T) and demands >> they be equal. In this case, the actual allocated type of x is Root, >> a proper subtype of T. >> >> For ISTYPE, it does call CheckIsType even when the types are opaque. >> >> The fact that it happens only when the types are opaque lends >> credibility to the fact that this bug has gone unnoticed at least >> as far back as PM3. > From hosking at cs.purdue.edu Thu Jan 23 17:51:02 2014 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 23 Jan 2014 11:51:02 -0500 Subject: [M3devel] EXT: proposal for getting wow64 context/stack reliably? In-Reply-To: References: Message-ID: <4A941D84-FC5A-499A-8F7F-DE06CEFC0B37@cs.purdue.edu> Yes, agreed. We need to address the native thread problem on all platforms (both pthread and windows). However, this particular problem is pernicious if we need to save M3 stack state at all external calls. Are there any linker tricks for Windows one can use to wrap only system calls? I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped, This approach is used for all modern Java implementations that I am aware of. It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Mobile +1 765 427 5484 On Jan 23, 2014, at 6:59 AM, Coleburn, Randy wrote: > Jay, > > If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. > > Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. > > Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. > > Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows? > > --Randy > > Sent from my iPhone > > On Jan 23, 2014, at 2:58 AM, "Jay K" wrote: > >> There is a behavior/bug in wow64. >> Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc. >> >> >> >> SuspendThread / GetThreadContext work like this: >> >> >> 32bit processes consist almost entirely of 32bit code. >> There is a small amount of 64bit code. >> >> >> If you suspend while running 32bit code, GetThreadContext works. >> if you suspend while running 64bit code, GetThreadContext usually but not always works. >> >> >> 64bit code is run en route to syscalls. >> For example you call: >> 1 kernel32!Sleep >> 2 it calls 32bit NtDelayExecution >> 3 that calls wow64NtDelayExecutation (via a cross segment "far" jmp or call) >> 4 which calls native NtDelayExecuation >> >> >> In between 2 and 3, within 64bit code, the 32bit context is saved. >> You can step through it very easily in a debugger. Really. >> Where GetThreadContext knows where to get it. >> The problem is that saving context is not atomic. >> You can suspend while saving context. >> >> >> What to do? >> >> >> scratch/wow64stack contains a program that detects the bug. >> I believe it is the basis of a workaround for the bug. >> >> >> Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, >> not only functions that use exception handling, but also functions that call "extern" functions >> call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection >> use that, if it isn't zero. >> Normally it will be zero. >> Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile. >> That is why it isn't a set/set-zero pattern (like in the test program). >> >> >> Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. >> >> >> Rejected counter proposal: >> Don't suspend/gc when running a syscall. >> No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that. >> >> >> Possible augmentation: if running native, short circuit most of this >> >> >> Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate >> compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. >> >> >> AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, >> until/unless there is another 32bit platform runable on some similar 64bit platforms. >> >> >> Performance impact: hypothetically large but probably not noticable. >> >> >> Furthe refinements: It isn't extern/native code per se, it is syscalls. >> We could further augment pragmas to discern them. >> >> >> We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls >> to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. >> It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... >> >> >> >> Agreed? I'll make the compiler change? >> >> >> Oh, also, not just stack pointer, but other registers, at least non-volatiles? >> >> >> Eventually cooperative suspend will cause this to fall away as a problem.. >> >> >> - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jan 23 21:03:09 2014 From: jay.krell at cornell.edu (Jay K) Date: Thu, 23 Jan 2014 20:03:09 +0000 Subject: [M3devel] EXT: proposal for getting wow64 context/stack reliably? In-Reply-To: References: , , <4A941D84-FC5A-499A-8F7F-DE06CEFC0B37@cs.purdue.edu>, Message-ID: Syscalls are "doubly wrapped". Exports from ntdll.dll that start "Nt" or "Zw" are syscalls and nothing else. If you look at them, they all look special. But they expose a standard calling convention. Lesser known, there are also some syscalls in gdi32.dll and user32.dll. None of them are public interfaces. They are all reached through public interfaces typically in kernel32.dll, gdi32.dll, user32.dll. For example, kernel32!CreateFileW might eventually call ntdll!NtCreateFile, but not directly, and this can change. Unix is similar and different -- sometimes functions are just a syscall, sometimes they are wrappers that do other work. There are counter examples -- functions that just return global variables like GetTickCount/GetTickCount64/GetCommandLine/GetEnvironmentVariable without a syscall. Bottom line is though, one isn't really supposed to know where the syscalls are. - Jay From: jay.krell at cornell.edu To: hosking at cs.purdue.edu; rcolebur at scires.com CC: m3devel at elegosoft.com Subject: RE: [M3devel] EXT: proposal for getting wow64 context/stack reliably? Date: Thu, 23 Jan 2014 19:57:27 +0000 There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same. Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok. - Jay From: hosking at cs.purdue.edu Date: Thu, 23 Jan 2014 11:51:02 -0500 To: rcolebur at SCIRES.COM CC: m3devel at elegosoft.com; jay.krell at cornell.edu Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack reliably? Yes, agreed.We need to address the native thread problem on all platforms (both pthread and windows). However, this particular problem is pernicious if we need to save M3 stack state at all external calls. Are there any linker tricks for Windows one can use to wrap only system calls? I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped, This approach is used for all modern Java implementations that I am aware of. It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state. Antony Hosking | Associate Professor | Computer Science | Purdue University305 N. University Street | West Lafayette | IN 47907 | USAMobile +1 765 427 5484 On Jan 23, 2014, at 6:59 AM, Coleburn, Randy wrote:Jay, If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows? --Randy Sent from my iPhone On Jan 23, 2014, at 2:58 AM, "Jay K" wrote: There is a behavior/bug in wow64. Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc. SuspendThread / GetThreadContext work like this: 32bit processes consist almost entirely of 32bit code. There is a small amount of 64bit code. If you suspend while running 32bit code, GetThreadContext works. if you suspend while running 64bit code, GetThreadContext usually but not always works. 64bit code is run en route to syscalls. For example you call: 1 kernel32!Sleep 2 it calls 32bit NtDelayExecution 3 that calls wow64NtDelayExecutation (via a cross segment "far" jmp or call) 4 which calls native NtDelayExecuation In between 2 and 3, within 64bit code, the 32bit context is saved. You can step through it very easily in a debugger. Really. Where GetThreadContext knows where to get it. The problem is that saving context is not atomic. You can suspend while saving context. What to do? scratch/wow64stack contains a program that detects the bug. I believe it is the basis of a workaround for the bug. Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, not only functions that use exception handling, but also functions that call "extern" functions call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection use that, if it isn't zero. Normally it will be zero. Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile. That is why it isn't a set/set-zero pattern (like in the test program). Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. Rejected counter proposal: Don't suspend/gc when running a syscall. No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that. Possible augmentation: if running native, short circuit most of this Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, until/unless there is another 32bit platform runable on some similar 64bit platforms. Performance impact: hypothetically large but probably not noticable. Furthe refinements: It isn't extern/native code per se, it is syscalls. We could further augment pragmas to discern them. We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... Agreed? I'll make the compiler change? Oh, also, not just stack pointer, but other registers, at least non-volatiles? Eventually cooperative suspend will cause this to fall away as a problem.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jan 23 20:57:27 2014 From: jay.krell at cornell.edu (Jay K) Date: Thu, 23 Jan 2014 19:57:27 +0000 Subject: [M3devel] EXT: proposal for getting wow64 context/stack reliably? In-Reply-To: <4A941D84-FC5A-499A-8F7F-DE06CEFC0B37@cs.purdue.edu> References: , , <4A941D84-FC5A-499A-8F7F-DE06CEFC0B37@cs.purdue.edu> Message-ID: There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same. Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok. - Jay From: hosking at cs.purdue.edu Date: Thu, 23 Jan 2014 11:51:02 -0500 To: rcolebur at SCIRES.COM CC: m3devel at elegosoft.com; jay.krell at cornell.edu Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack reliably? Yes, agreed.We need to address the native thread problem on all platforms (both pthread and windows). However, this particular problem is pernicious if we need to save M3 stack state at all external calls. Are there any linker tricks for Windows one can use to wrap only system calls? I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped, This approach is used for all modern Java implementations that I am aware of. It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state. Antony Hosking | Associate Professor | Computer Science | Purdue University305 N. University Street | West Lafayette | IN 47907 | USAMobile +1 765 427 5484 On Jan 23, 2014, at 6:59 AM, Coleburn, Randy wrote:Jay, If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows? --Randy Sent from my iPhone On Jan 23, 2014, at 2:58 AM, "Jay K" wrote: There is a behavior/bug in wow64. Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc. SuspendThread / GetThreadContext work like this: 32bit processes consist almost entirely of 32bit code. There is a small amount of 64bit code. If you suspend while running 32bit code, GetThreadContext works. if you suspend while running 64bit code, GetThreadContext usually but not always works. 64bit code is run en route to syscalls. For example you call: 1 kernel32!Sleep 2 it calls 32bit NtDelayExecution 3 that calls wow64NtDelayExecutation (via a cross segment "far" jmp or call) 4 which calls native NtDelayExecuation In between 2 and 3, within 64bit code, the 32bit context is saved. You can step through it very easily in a debugger. Really. Where GetThreadContext knows where to get it. The problem is that saving context is not atomic. You can suspend while saving context. What to do? scratch/wow64stack contains a program that detects the bug. I believe it is the basis of a workaround for the bug. Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, not only functions that use exception handling, but also functions that call "extern" functions call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection use that, if it isn't zero. Normally it will be zero. Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile. That is why it isn't a set/set-zero pattern (like in the test program). Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. Rejected counter proposal: Don't suspend/gc when running a syscall. No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that. Possible augmentation: if running native, short circuit most of this Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, until/unless there is another 32bit platform runable on some similar 64bit platforms. Performance impact: hypothetically large but probably not noticable. Furthe refinements: It isn't extern/native code per se, it is syscalls. We could further augment pragmas to discern them. We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... Agreed? I'll make the compiler change? Oh, also, not just stack pointer, but other registers, at least non-volatiles? Eventually cooperative suspend will cause this to fall away as a problem.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcolebur at SCIRES.COM Thu Jan 23 23:23:43 2014 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Thu, 23 Jan 2014 22:23:43 +0000 Subject: [M3devel] proposal for getting wow64 context/stack reliably? Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL> Jay: When you say native 32/64-bit "should be ok", you really mean ok with respect to the SYSWOW64 problem you've identified. Both of these native implementations still have threading misbehaviors as identified by the thread test program. Correct? --Randy From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Thursday, January 23, 2014 2:57 PM To: Tony; Coleburn, Randy Cc: m3devel Subject: EXT:RE: [M3devel] EXT: proposal for getting wow64 context/stack reliably? There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same. Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok. - Jay ________________________________ From: hosking at cs.purdue.edu Date: Thu, 23 Jan 2014 11:51:02 -0500 To: rcolebur at SCIRES.COM CC: m3devel at elegosoft.com; jay.krell at cornell.edu Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack reliably? Yes, agreed. We need to address the native thread problem on all platforms (both pthread and windows). However, this particular problem is pernicious if we need to save M3 stack state at all external calls. Are there any linker tricks for Windows one can use to wrap only system calls? I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped, This approach is used for all modern Java implementations that I am aware of. It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Mobile +1 765 427 5484 On Jan 23, 2014, at 6:59 AM, Coleburn, Randy > wrote: Jay, If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows? --Randy Sent from my iPhone On Jan 23, 2014, at 2:58 AM, "Jay K" > wrote: There is a behavior/bug in wow64. Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc. SuspendThread / GetThreadContext work like this: 32bit processes consist almost entirely of 32bit code. There is a small amount of 64bit code. If you suspend while running 32bit code, GetThreadContext works. if you suspend while running 64bit code, GetThreadContext usually but not always works. 64bit code is run en route to syscalls. For example you call: 1 kernel32!Sleep 2 it calls 32bit NtDelayExecution 3 that calls wow64NtDelayExecutation (via a cross segment "far" jmp or call) 4 which calls native NtDelayExecuation In between 2 and 3, within 64bit code, the 32bit context is saved. You can step through it very easily in a debugger. Really. Where GetThreadContext knows where to get it. The problem is that saving context is not atomic. You can suspend while saving context. What to do? scratch/wow64stack contains a program that detects the bug. I believe it is the basis of a workaround for the bug. Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, not only functions that use exception handling, but also functions that call "extern" functions call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection use that, if it isn't zero. Normally it will be zero. Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile. That is why it isn't a set/set-zero pattern (like in the test program). Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. Rejected counter proposal: Don't suspend/gc when running a syscall. No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that. Possible augmentation: if running native, short circuit most of this Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, until/unless there is another 32bit platform runable on some similar 64bit platforms. Performance impact: hypothetically large but probably not noticable. Furthe refinements: It isn't extern/native code per se, it is syscalls. We could further augment pragmas to discern them. We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... Agreed? I'll make the compiler change? Oh, also, not just stack pointer, but other registers, at least non-volatiles? Eventually cooperative suspend will cause this to fall away as a problem.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Jan 24 00:09:50 2014 From: jay.krell at cornell.edu (Jay K) Date: Thu, 23 Jan 2014 23:09:50 +0000 Subject: [M3devel] proposal for getting wow64 context/stack reliably? In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL> References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL> Message-ID: Yes. Agreed. They should be ok with respect to SuspendThread + GetThreadContext. The small C++ test program in scratch. I don't know about the "other" problem that affects even pthreads and I agree we should focus on that first, on pthreads, native I386_NT, and possibly AMD64_NT, but for now, at least briefly, ignore I386_NT on 64bit Windows. - Jay From: rcolebur at SCIRES.COM To: m3devel at elegosoft.com Date: Thu, 23 Jan 2014 22:23:43 +0000 Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably? Jay: When you say native 32/64-bit ?should be ok?, you really mean ok with respect to the SYSWOW64 problem you?ve identified. Both of these native implementations still have threading misbehaviors as identified by the thread test program. Correct? --Randy From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Thursday, January 23, 2014 2:57 PM To: Tony; Coleburn, Randy Cc: m3devel Subject: EXT:RE: [M3devel] EXT: proposal for getting wow64 context/stack reliably? There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same. Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok. - Jay From: hosking at cs.purdue.edu Date: Thu, 23 Jan 2014 11:51:02 -0500 To: rcolebur at SCIRES.COM CC: m3devel at elegosoft.com; jay.krell at cornell.edu Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack reliably? Yes, agreed. We need to address the native thread problem on all platforms (both pthread and windows). However, this particular problem is pernicious if we need to save M3 stack state at all external calls. Are there any linker tricks for Windows one can use to wrap only system calls? I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped, This approach is used for all modern Java implementations that I am aware of. It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Mobile +1 765 427 5484 On Jan 23, 2014, at 6:59 AM, Coleburn, Randy wrote: Jay, If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows? --Randy Sent from my iPhone On Jan 23, 2014, at 2:58 AM, "Jay K" wrote: There is a behavior/bug in wow64. Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc. SuspendThread / GetThreadContext work like this: 32bit processes consist almost entirely of 32bit code. There is a small amount of 64bit code. If you suspend while running 32bit code, GetThreadContext works. if you suspend while running 64bit code, GetThreadContext usually but not always works. 64bit code is run en route to syscalls. For example you call: 1 kernel32!Sleep 2 it calls 32bit NtDelayExecution 3 that calls wow64NtDelayExecutation (via a cross segment "far" jmp or call) 4 which calls native NtDelayExecuation In between 2 and 3, within 64bit code, the 32bit context is saved. You can step through it very easily in a debugger. Really. Where GetThreadContext knows where to get it. The problem is that saving context is not atomic. You can suspend while saving context. What to do? scratch/wow64stack contains a program that detects the bug. I believe it is the basis of a workaround for the bug. Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, not only functions that use exception handling, but also functions that call "extern" functions call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection use that, if it isn't zero. Normally it will be zero. Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile. That is why it isn't a set/set-zero pattern (like in the test program). Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. Rejected counter proposal: Don't suspend/gc when running a syscall. No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that. Possible augmentation: if running native, short circuit most of this Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, until/unless there is another 32bit platform runable on some similar 64bit platforms. Performance impact: hypothetically large but probably not noticable. Furthe refinements: It isn't extern/native code per se, it is syscalls. We could further augment pragmas to discern them. We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... Agreed? I'll make the compiler change? Oh, also, not just stack pointer, but other registers, at least non-volatiles? Eventually cooperative suspend will cause this to fall away as a problem.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Fri Jan 24 00:42:01 2014 From: adacore at marino.st (John Marino) Date: Fri, 24 Jan 2014 00:42:01 +0100 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: <52DBCDBA.8030809@marino.st> References: <52DBA2EF.4020101@marino.st>, , <52DBC081.1070401@marino.st> , <52DBC748.5090308@marino.st> <52DBCDBA.8030809@marino.st> Message-ID: <52E1A8C9.3010903@marino.st> On 1/19/2014 14:06, John Marino wrote: > On 1/19/2014 13:50, Jay K wrote: >> And there are advantages not yet implemented, in particular, efficient >> exception handling by generating C++. > > Like FreeBSD (and unlike NetBSD and OpenBSD), DragonFly has implemented > dl_iterate_phdr which gcc uses for efficient zero-cost exception > handling. That might come in handy here (I implemented it in > DragonFly's real-time linker, mainly for Ada/GNAT but C++ benefited.) > > >> Your patches should apply fairly cleanly to any new checkout. >> I am willing to through them and apply them to CVS if you want -- >> depends on if you are ok with my stealing your credit, vs. you want to >> commit them yourselves. > > I would love it if you did this - I'm happy with a mention in the commit > message. I just attached a tarball of all the patches I'm using right > now. Some of these changes are specifically for the cross compiler (I'm > think of the /scripts changes mostly) but most are valid assuming they > still apply to the head of the repo. Please take as much as you want > with my gratitude. > >> I already skimmed them and they all look easy. >> DragonflyBSD is basically FreeBSD by another name. > > That's not an accident. As a DragonFly developer that mostly works on > userland, I've been converging FreeBSD and DragonFly again. They > drifted apart after 10 years, but I thought it better to keep the > userlands as similar as possible for 3rd party applications. > >> Yes, a ton of work has been done on the kernel, file systems, maybe ports. >> But the ABI is almost the same -- heck, for the most part, >> Linux==NetBSD==FreeBSD==OpenBSD. >> Sure, they might be implemented differently, but they are highly source >> compatible and every highly object code compatible. >>> I must be misunderstanding what you mean by "upgrade". It's the >>> cross-compiler that's complaining, and it was built by the latest patches. >> Agreed, we are both missing something simple. >> Usually this is the error you get when you don't upgrade >> Target.i3/Target.m3. >> If you want to press on with the current approach and ignore my advise, > > I'm not ignoring it; it's more like I'm not quite ready to give up since > I'm so close. I think I could actually get these libs and cm3 to build > on DragonFly if I used AMD64_FREEBSD as the target with the > cm3cg-AMD_DRAGONFLY generator. It was building before. I just figured > I'd then get stuck on the DragonFly side trying to compile cm3 with > itself, theoretically I should get the same error there. > > >> Ugh. I saw something about that in your diffs. Definitely we intend to >> link statically. >> I actually went to the trouble of removing use of gmp/mpfr/mpc in >> current source. >> They are used for actually very little and aren't worth it. > Hi Jay, If you get those patches committed and publish an official snapshot (compressed tarball) with those changes in it, then I'll update the FreeBSD port to that snapshot. If new patches are required, say to convert FreeBSD to use the C backend, I'll feed those back too. Then I'll try porting DragonFly again, your way. Thanks, John From jay.krell at cornell.edu Fri Jan 24 09:45:39 2014 From: jay.krell at cornell.edu (Jay K) Date: Fri, 24 Jan 2014 08:45:39 +0000 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: <52E1A8C9.3010903@marino.st> References: <52DBA2EF.4020101@marino.st>, , <52DBC081.1070401@marino.st> , <52DBC748.5090308@marino.st> <52DBCDBA.8030809@marino.st>,<52E1A8C9.3010903@marino.st> Message-ID: sorry, I'm delayed..so much to do.. I commited the networking fix.. The m3core/unix one I think we already have. Later... - Jay > Date: Fri, 24 Jan 2014 00:42:01 +0100 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) > > On 1/19/2014 14:06, John Marino wrote: > > On 1/19/2014 13:50, Jay K wrote: > >> And there are advantages not yet implemented, in particular, efficient > >> exception handling by generating C++. > > > > Like FreeBSD (and unlike NetBSD and OpenBSD), DragonFly has implemented > > dl_iterate_phdr which gcc uses for efficient zero-cost exception > > handling. That might come in handy here (I implemented it in > > DragonFly's real-time linker, mainly for Ada/GNAT but C++ benefited.) > > > > > >> Your patches should apply fairly cleanly to any new checkout. > >> I am willing to through them and apply them to CVS if you want -- > >> depends on if you are ok with my stealing your credit, vs. you want to > >> commit them yourselves. > > > > I would love it if you did this - I'm happy with a mention in the commit > > message. I just attached a tarball of all the patches I'm using right > > now. Some of these changes are specifically for the cross compiler (I'm > > think of the /scripts changes mostly) but most are valid assuming they > > still apply to the head of the repo. Please take as much as you want > > with my gratitude. > > > >> I already skimmed them and they all look easy. > >> DragonflyBSD is basically FreeBSD by another name. > > > > That's not an accident. As a DragonFly developer that mostly works on > > userland, I've been converging FreeBSD and DragonFly again. They > > drifted apart after 10 years, but I thought it better to keep the > > userlands as similar as possible for 3rd party applications. > > > >> Yes, a ton of work has been done on the kernel, file systems, maybe ports. > >> But the ABI is almost the same -- heck, for the most part, > >> Linux==NetBSD==FreeBSD==OpenBSD. > >> Sure, they might be implemented differently, but they are highly source > >> compatible and every highly object code compatible. > >>> I must be misunderstanding what you mean by "upgrade". It's the > >>> cross-compiler that's complaining, and it was built by the latest patches. > >> Agreed, we are both missing something simple. > >> Usually this is the error you get when you don't upgrade > >> Target.i3/Target.m3. > >> If you want to press on with the current approach and ignore my advise, > > > > I'm not ignoring it; it's more like I'm not quite ready to give up since > > I'm so close. I think I could actually get these libs and cm3 to build > > on DragonFly if I used AMD64_FREEBSD as the target with the > > cm3cg-AMD_DRAGONFLY generator. It was building before. I just figured > > I'd then get stuck on the DragonFly side trying to compile cm3 with > > itself, theoretically I should get the same error there. > > > > > >> Ugh. I saw something about that in your diffs. Definitely we intend to > >> link statically. > >> I actually went to the trouble of removing use of gmp/mpfr/mpc in > >> current source. > >> They are used for actually very little and aren't worth it. > > > > Hi Jay, > If you get those patches committed and publish an official snapshot > (compressed tarball) with those changes in it, then I'll update the > FreeBSD port to that snapshot. > > If new patches are required, say to convert FreeBSD to use the C > backend, I'll feed those back too. > > Then I'll try porting DragonFly again, your way. > Thanks, > John > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Fri Jan 24 10:11:02 2014 From: adacore at marino.st (John Marino) Date: Fri, 24 Jan 2014 10:11:02 +0100 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: References: <52DBA2EF.4020101@marino.st>, , <52DBC081.1070401@marino.st> , <52DBC748.5090308@marino.st> <52DBCDBA.8030809@marino.st>, <52E1A8C9.3010903@marino.st> Message-ID: <52E22E26.10402@marino.st> On 1/24/2014 09:45, Jay K wrote: > sorry, I'm delayed..so much to do.. > I commited the networking fix.. > The m3core/unix one I think we already have. > > Later... > - Jay Right, but there's no downloadable source snapshot. And if the DragonFly support isn't in there, I'll have to carry all those patches too. So my goal is to have a few patches as possible. Anyway, I have to base the port on something -- pulling latest SVN is rarely done, and CVS is never done. Some github projects don't make releases and needed to be pulled directly, but this requires first git to be built (lots of dependencies) and then github tweaks their service and everything fails. So I'm a big believer in static compressed tarballs. Which means I can't update FreeBSD cm3 version until one exists. theoretically I can package one myself, but it looks better if it's "official" and comes from one of the known m3 websites. Thanks, John From jay.krell at cornell.edu Fri Jan 24 10:59:24 2014 From: jay.krell at cornell.edu (Jay K) Date: Fri, 24 Jan 2014 09:59:24 +0000 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: <52E22E26.10402@marino.st> References: <52DBA2EF.4020101@marino.st>, , <52DBC081.1070401@marino.st> , <52DBC748.5090308@marino.st> <52DBCDBA.8030809@marino.st>,<52E1A8C9.3010903@marino.st> , <52E22E26.10402@marino.st> Message-ID: Please be patient. We'll get a snapshot, soon. A release, unclear. - Jay > Date: Fri, 24 Jan 2014 10:11:02 +0100 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) > > On 1/24/2014 09:45, Jay K wrote: > > sorry, I'm delayed..so much to do.. > > I commited the networking fix.. > > The m3core/unix one I think we already have. > > > > Later... > > - Jay > > > Right, but there's no downloadable source snapshot. And if the > DragonFly support isn't in there, I'll have to carry all those patches > too. So my goal is to have a few patches as possible. > > Anyway, I have to base the port on something -- pulling latest SVN is > rarely done, and CVS is never done. Some github projects don't make > releases and needed to be pulled directly, but this requires first git > to be built (lots of dependencies) and then github tweaks their service > and everything fails. So I'm a big believer in static compressed > tarballs. > > Which means I can't update FreeBSD cm3 version until one exists. > theoretically I can package one myself, but it looks better if it's > "official" and comes from one of the known m3 websites. > > Thanks, > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Jan 24 11:38:01 2014 From: jay.krell at cornell.edu (Jay K) Date: Fri, 24 Jan 2014 10:38:01 +0000 Subject: [M3devel] garbage collector compacting and volatile? Message-ID: Is our garbage collector compacting? In particular, what model do we need for "volatile" in codegen? Does the compactor read the registers or just the stack? Update references anywhere? Just the stack? Also context? There are tradeoffs either way. If it is not compacting and only reads the stack, then it can be more portable, but every write of a collected type/pointer would have to be volatile. Or is that what the barriers are for? If we update values on the stack, then there is some portability, but reads and writes would have to be volatile. If we read and write registers in the collector, then nothing has to be volatile. Should this all be parameters passed to the backend? And/or preserved via #ifdef in the C backend? That last part is easy enough, for generated C to take #ifdefs indicating if reads and/or writes should be volatile. If the collector is not compacting, then it need not update stack or register values, just possibly read them. I recall reading that it is compacting. Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Fri Jan 24 19:24:35 2014 From: hosking at cs.purdue.edu (Tony Hosking) Date: Fri, 24 Jan 2014 13:24:35 -0500 Subject: [M3devel] proposal for getting wow64 context/stack reliably? In-Reply-To: References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL> Message-ID: As a way forward I propose that we indeed treat EXTERNAL calls as needing to capture a GC context. At least that should be the default. Some calls could be annotated as not needing to capture the GC context with a different EXTERNAL pragma. Knowing that such a call can safely ignore saving the context would require inspection of the external function, or understanding of its behavior. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Mobile +1 765 427 5484 On Jan 23, 2014, at 6:09 PM, Jay K wrote: > Yes. Agreed. > They should be ok with respect to SuspendThread + GetThreadContext. The small C++ test program in scratch. > I don't know about the "other" problem that affects even pthreads and I agree we should focus on that first, on pthreads, native I386_NT, and possibly AMD64_NT, but for now, at least briefly, ignore I386_NT on 64bit Windows. > > > - Jay > > > From: rcolebur at SCIRES.COM > To: m3devel at elegosoft.com > Date: Thu, 23 Jan 2014 22:23:43 +0000 > Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably? > > Jay: > > When you say native 32/64-bit ?should be ok?, you really mean ok with respect to the SYSWOW64 problem you?ve identified. > > Both of these native implementations still have threading misbehaviors as identified by the thread test program. Correct? > > --Randy > > From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K > Sent: Thursday, January 23, 2014 2:57 PM > To: Tony; Coleburn, Randy > Cc: m3devel > Subject: EXT:RE: [M3devel] EXT: proposal for getting wow64 context/stack reliably? > > There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same. > Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok. > > - Jay > From: hosking at cs.purdue.edu > Date: Thu, 23 Jan 2014 11:51:02 -0500 > To: rcolebur at SCIRES.COM > CC: m3devel at elegosoft.com; jay.krell at cornell.edu > Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack reliably? > > Yes, agreed. > We need to address the native thread problem on all platforms (both pthread and windows). > > However, this particular problem is pernicious if we need to save M3 stack state at all external calls. > > Are there any linker tricks for Windows one can use to wrap only system calls? > > I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped, This approach is used for all modern Java implementations that I am aware of. > > It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state. > > Antony Hosking | Associate Professor | Computer Science | Purdue University > 305 N. University Street | West Lafayette | IN 47907 | USA > Mobile +1 765 427 5484 > > > > > > > On Jan 23, 2014, at 6:59 AM, Coleburn, Randy wrote: > > Jay, > > If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. > > Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. > > Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. > > Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows? > > --Randy > > Sent from my iPhone > > On Jan 23, 2014, at 2:58 AM, "Jay K" wrote: > There is a behavior/bug in wow64. > Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc. > > > > SuspendThread / GetThreadContext work like this: > > > 32bit processes consist almost entirely of 32bit code. > There is a small amount of 64bit code. > > > If you suspend while running 32bit code, GetThreadContext works. > if you suspend while running 64bit code, GetThreadContext usually but not always works. > > > 64bit code is run en route to syscalls. > For example you call: > 1 kernel32!Sleep > 2 it calls 32bit NtDelayExecution > 3 that calls wow64NtDelayExecutation (via a cross segment "far" jmp or call) > 4 which calls native NtDelayExecuation > > > In between 2 and 3, within 64bit code, the 32bit context is saved. > You can step through it very easily in a debugger. Really. > Where GetThreadContext knows where to get it. > The problem is that saving context is not atomic. > You can suspend while saving context. > > > What to do? > > > scratch/wow64stack contains a program that detects the bug. > I believe it is the basis of a workaround for the bug. > > > Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, > not only functions that use exception handling, but also functions that call "extern" functions > call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection > use that, if it isn't zero. > Normally it will be zero. > Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile. > That is why it isn't a set/set-zero pattern (like in the test program). > > > Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. > > > Rejected counter proposal: > Don't suspend/gc when running a syscall. > No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that. > > > Possible augmentation: if running native, short circuit most of this > > > Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate > compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. > > > AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, > until/unless there is another 32bit platform runable on some similar 64bit platforms. > > > Performance impact: hypothetically large but probably not noticable. > > > Furthe refinements: It isn't extern/native code per se, it is syscalls. > We could further augment pragmas to discern them. > > > We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls > to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. > It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... > > > > Agreed? I'll make the compiler change? > > > Oh, also, not just stack pointer, but other registers, at least non-volatiles? > > > Eventually cooperative suspend will cause this to fall away as a problem.. > > > - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Fri Jan 24 19:12:33 2014 From: hosking at cs.purdue.edu (Tony Hosking) Date: Fri, 24 Jan 2014 13:12:33 -0500 Subject: [M3devel] garbage collector compacting and volatile? In-Reply-To: References: Message-ID: <8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu> On Jan 24, 2014, at 5:38 AM, Jay K wrote: > Is our garbage collector compacting? Yes, it is mostly-copying. Only pages that are not referred to from the stack (i.e., only from other objects) have their objects relocated (compacted). > In particular, what model do we need for "volatile" in codegen? > Does the compactor read the registers or just the stack? It reads both registers and stack. > Update references anywhere? Just the stack? Also context? The collector first pins all pages referenced from the stacks, including registers. Any page not pinned has its reachable objects evacuated and all references to those objects (i.e., from other objects) are updated to refer to the the new location. > There are tradeoffs either way. Indeed. Too many to enumerate here. > If it is not compacting and only reads the stack, then it can be more > portable, but every write of a collected type/pointer would have to be volatile. > Or is that what the barriers are for? The write barrier allows the collector to run concurrently with the mutator (once all stack roots have been sampled). > If we update values on the stack, then there is some portability, > but reads and writes would have to be volatile. No updates to values on the stacks. > If we read and write registers in the collector, then nothing has to be volatile. Read only. No writes to registers. > Should this all be parameters passed to the backend? Nope. > And/or preserved via #ifdef in the C backend? > That last part is easy enough, for generated C to take #ifdefs > indicating if reads and/or writes should be volatile. Nope. > If the collector is not compacting, then it need not update > stack or register values, just possibly read them. > > I recall reading that it is compacting. It is mostly-copying. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sat Jan 25 05:04:59 2014 From: jay.krell at cornell.edu (Jay K) Date: Sat, 25 Jan 2014 04:04:59 +0000 Subject: [M3devel] proposal for getting wow64 context/stack reliably? In-Reply-To: References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>, , Message-ID: I might yet have another solution here..there might be a way to detect the race and retry...later.. - Jay From: hosking at cs.purdue.edu Date: Fri, 24 Jan 2014 13:24:35 -0500 To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably? As a way forward I propose that we indeed treat EXTERNAL calls as needing to capture a GC context. At least that should be the default.Some calls could be annotated as not needing to capture the GC context with a different EXTERNAL pragma. Knowing that such a call can safely ignore saving the context would require inspection of the external function, or understanding of its behavior. Antony Hosking | Associate Professor | Computer Science | Purdue University305 N. University Street | West Lafayette | IN 47907 | USAMobile +1 765 427 5484 On Jan 23, 2014, at 6:09 PM, Jay K wrote:Yes. Agreed. They should be ok with respect to SuspendThread + GetThreadContext. The small C++ test program in scratch. I don't know about the "other" problem that affects even pthreads and I agree we should focus on that first, on pthreads, native I386_NT, and possibly AMD64_NT, but for now, at least briefly, ignore I386_NT on 64bit Windows. - Jay From: rcolebur at SCIRES.COM To: m3devel at elegosoft.com Date: Thu, 23 Jan 2014 22:23:43 +0000 Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably? Jay: When you say native 32/64-bit ?should be ok?, you really mean ok with respect to the SYSWOW64 problem you?ve identified. Both of these native implementations still have threading misbehaviors as identified by the thread test program. Correct? --Randy From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Thursday, January 23, 2014 2:57 PM To: Tony; Coleburn, Randy Cc: m3devel Subject: EXT:RE: [M3devel] EXT: proposal for getting wow64 context/stack reliably? There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same. Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok. - JayFrom: hosking at cs.purdue.edu Date: Thu, 23 Jan 2014 11:51:02 -0500 To: rcolebur at SCIRES.COM CC: m3devel at elegosoft.com; jay.krell at cornell.edu Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack reliably? Yes, agreed.We need to address the native thread problem on all platforms (both pthread and windows). However, this particular problem is pernicious if we need to save M3 stack state at all external calls. Are there any linker tricks for Windows one can use to wrap only system calls? I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped, This approach is used for all modern Java implementations that I am aware of. It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state. Antony Hosking | Associate Professor | Computer Science | Purdue University305 N. University Street | West Lafayette | IN 47907 | USAMobile +1 765 427 5484 On Jan 23, 2014, at 6:59 AM, Coleburn, Randy wrote: Jay, If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows? --Randy Sent from my iPhone On Jan 23, 2014, at 2:58 AM, "Jay K" wrote: There is a behavior/bug in wow64. Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc. SuspendThread / GetThreadContext work like this: 32bit processes consist almost entirely of 32bit code. There is a small amount of 64bit code. If you suspend while running 32bit code, GetThreadContext works. if you suspend while running 64bit code, GetThreadContext usually but not always works. 64bit code is run en route to syscalls. For example you call: 1 kernel32!Sleep 2 it calls 32bit NtDelayExecution 3 that calls wow64NtDelayExecutation (via a cross segment "far" jmp or call) 4 which calls native NtDelayExecuation In between 2 and 3, within 64bit code, the 32bit context is saved. You can step through it very easily in a debugger. Really. Where GetThreadContext knows where to get it. The problem is that saving context is not atomic. You can suspend while saving context. What to do? scratch/wow64stack contains a program that detects the bug. I believe it is the basis of a workaround for the bug. Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, not only functions that use exception handling, but also functions that call "extern" functions call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection use that, if it isn't zero. Normally it will be zero. Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile. That is why it isn't a set/set-zero pattern (like in the test program). Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. Rejected counter proposal: Don't suspend/gc when running a syscall. No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that. Possible augmentation: if running native, short circuit most of this Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, until/unless there is another 32bit platform runable on some similar 64bit platforms. Performance impact: hypothetically large but probably not noticable. Furthe refinements: It isn't extern/native code per se, it is syscalls. We could further augment pragmas to discern them. We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... Agreed? I'll make the compiler change? Oh, also, not just stack pointer, but other registers, at least non-volatiles? Eventually cooperative suspend will cause this to fall away as a problem.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Sat Jan 25 05:38:03 2014 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Fri, 24 Jan 2014 23:38:03 -0500 Subject: [M3devel] proposal for getting wow64 context/stack reliably? In-Reply-To: References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL> Message-ID: <20140125043803.GA3423@topoi.pooq.com> On Sat, Jan 25, 2014 at 04:04:59AM +0000, Jay K wrote: > I might yet have another solution here..there might be a way to > detect the race and retry...later.. > > - Jay I seem to remember a lot of comments in the trestle code tht seem to be intended as assertions to catch race conditions. Is any of that stuff conceivably helpful here? -- hendrik From jay.krell at cornell.edu Sun Jan 26 07:08:20 2014 From: jay.krell at cornell.edu (Jay K) Date: Sun, 26 Jan 2014 06:08:20 +0000 Subject: [M3devel] proposal for getting wow64 context/stack reliably? In-Reply-To: <20140125043803.GA3423@topoi.pooq.com> References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>, , , , <20140125043803.GA3423@topoi.pooq.com> Message-ID: No. > Date: Fri, 24 Jan 2014 23:38:03 -0500 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably? > > On Sat, Jan 25, 2014 at 04:04:59AM +0000, Jay K wrote: > > I might yet have another solution here..there might be a way to > > detect the race and retry...later.. > > > > - Jay > > I seem to remember a lot of comments in the trestle code tht seem to > be intended as assertions to catch race conditions. > Is any of that stuff conceivably helpful here? > > -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Jan 27 08:23:26 2014 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jan 2014 07:23:26 +0000 Subject: [M3devel] proposal for getting wow64 context/stack reliably? In-Reply-To: References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>, , , , , , , , <20140125043803.GA3423@topoi.pooq.com>, Message-ID: > I might yet have another solution here And it might only be for Windows 8 or newer.I tend to think we should go with the earlier proposal -- save context ourselves in thread locals when calling externals. Likely portable implementation is setjmp or getcontext, though we can maybe do better.This would be part of cooperative suspend anyway, which we want to move to. - Jay From: jay.krell at cornell.edu To: hendrik at topoi.pooq.com; m3devel at elegosoft.com Date: Sun, 26 Jan 2014 06:08:20 +0000 Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably? No. > Date: Fri, 24 Jan 2014 23:38:03 -0500 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably? > > On Sat, Jan 25, 2014 at 04:04:59AM +0000, Jay K wrote: > > I might yet have another solution here..there might be a way to > > detect the race and retry...later.. > > > > - Jay > > I seem to remember a lot of comments in the trestle code tht seem to > be intended as assertions to catch race conditions. > Is any of that stuff conceivably helpful here? > > -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Jan 27 08:45:08 2014 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jan 2014 07:45:08 +0000 Subject: [M3devel] garbage collector compacting and volatile? In-Reply-To: <8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu> References: , <8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu> Message-ID: In summary, shortest: We must be able to read registers (or save them to the stack or elsewhere (known thread locals)). We must be able to read the stack. We need not write registers or stack (from garbage collector). Slight elaboration: Copying/compaction only occurs when references are only from globals and heap. Therefore updates/writes only occur to globals and heap. Is there any desire to do better/different? To require the ability to update stack/registers? What does Java typically to? I guess there is the problem that we don't know for certain if values in registers and stack are pointers, or just coincident integers, and so updating them can be wrong.Whereas in the globals and "reachable thereof" we have precise type information for, so we can update those values safely.To do better/different would require more backend cooperation.We conservatively treat registers/stack as pointers if they look like them, and therefore keep values alive, but not trust them so much as to update them. Thanks, - Jay From: hosking at cs.purdue.edu Date: Fri, 24 Jan 2014 13:12:33 -0500 To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] garbage collector compacting and volatile? On Jan 24, 2014, at 5:38 AM, Jay K wrote: Is our garbage collector compacting? Yes, it is mostly-copying.Only pages that are not referred to from the stack (i.e., only from other objects) have their objects relocated (compacted). In particular, what model do we need for "volatile" in codegen? Does the compactor read the registers or just the stack? It reads both registers and stack. Update references anywhere? Just the stack? Also context? The collector first pins all pages referenced from the stacks, including registers.Any page not pinned has its reachable objects evacuated and all references to those objects (i.e., from other objects) are updated to refer to the the new location. There are tradeoffs either way. Indeed. Too many to enumerate here. If it is not compacting and only reads the stack, then it can be more portable, but every write of a collected type/pointer would have to be volatile. Or is that what the barriers are for? The write barrier allows the collector to run concurrently with the mutator (once all stack roots have been sampled). If we update values on the stack, then there is some portability, but reads and writes would have to be volatile. No updates to values on the stacks. If we read and write registers in the collector, then nothing has to be volatile. Read only. No writes to registers. Should this all be parameters passed to the backend? Nope. And/or preserved via #ifdef in the C backend? That last part is easy enough, for generated C to take #ifdefs indicating if reads and/or writes should be volatile. Nope. If the collector is not compacting, then it need not update stack or register values, just possibly read them. I recall reading that it is compacting. It is mostly-copying. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Mon Jan 27 16:16:57 2014 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 27 Jan 2014 10:16:57 -0500 Subject: [M3devel] garbage collector compacting and volatile? In-Reply-To: References: , <8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu> Message-ID: <4798B41F-079F-463C-A4FB-C36185B29997@cs.purdue.edu> On Jan 27, 2014, at 2:45 AM, Jay K wrote: > In summary, shortest: > > > We must be able to read registers (or save them to the stack or elsewhere (known thread locals)). > We must be able to read the stack. > We need not write registers or stack (from garbage collector). Correct. > Slight elaboration: > Copying/compaction only occurs when references are only from globals and heap. > Therefore updates/writes only occur to globals and heap. Correct. > Is there any desire to do better/different? > To require the ability to update stack/registers? Without control of the register allocator, optimisations, and stack layout performed by the compiler (especially back-end) getting fully accurate stack maps is very difficult. > What does Java typically to? Java VMs usually control their own destiny, having full control of JIT and stack layout. viz. HotSpot, IBM J9, Jikes RVM. > I guess there is the problem that we don't know for certain if values in registers and stack are pointers, or just coincident integers, and so updating them can be wrong. Correct. > Whereas in the globals and "reachable thereof" we have precise type information for, so we can update those values safely. Correct. > To do better/different would require more backend cooperation. > We conservatively treat registers/stack as pointers if they look like them, and therefore keep values alive, but not trust them so much as to update them. Correct. > > > Thanks, > - Jay > > From: hosking at cs.purdue.edu > Date: Fri, 24 Jan 2014 13:12:33 -0500 > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] garbage collector compacting and volatile? > > On Jan 24, 2014, at 5:38 AM, Jay K wrote: > > Is our garbage collector compacting? > > Yes, it is mostly-copying. > Only pages that are not referred to from the stack (i.e., only from other objects) have their objects relocated (compacted). > > In particular, what model do we need for "volatile" in codegen? > Does the compactor read the registers or just the stack? > > It reads both registers and stack. > > Update references anywhere? Just the stack? Also context? > > The collector first pins all pages referenced from the stacks, including registers. > Any page not pinned has its reachable objects evacuated and all references to those objects (i.e., from other objects) are updated to refer to the the new location. > > There are tradeoffs either way. > > Indeed. Too many to enumerate here. > > If it is not compacting and only reads the stack, then it can be more > portable, but every write of a collected type/pointer would have to be volatile. > Or is that what the barriers are for? > > The write barrier allows the collector to run concurrently with the mutator (once all stack roots have been sampled). > > If we update values on the stack, then there is some portability, > but reads and writes would have to be volatile. > > No updates to values on the stacks. > > If we read and write registers in the collector, then nothing has to be volatile. > > Read only. No writes to registers. > > Should this all be parameters passed to the backend? > > Nope. > > And/or preserved via #ifdef in the C backend? > That last part is easy enough, for generated C to take #ifdefs > indicating if reads and/or writes should be volatile. > > Nope. > > If the collector is not compacting, then it need not update > stack or register values, just possibly read them. > > I recall reading that it is compacting. > > It is mostly-copying. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Mon Jan 27 20:46:23 2014 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 27 Jan 2014 13:46:23 -0600 Subject: [M3devel] New compile failure Message-ID: <52E6B78F.7000003@lcwb.coop> The compiler in the head, with some recent changes, gives: rodney at yellowstone:~/proj/m3/cm3-head/cm3/m3-sys/m3middle$ cm3 Fatal Error: configuration file didn't specify BUILD_DIR This message is coming from cm3/src/Main.m3, which has recently changed, but there are related changes to other source files too. Reverting Main.m3 to the previous version won't compile. Putting an older executable for cm3 into the bin directory, with recently updated and installed cm3.cfg and config works. From rodney_bates at lcwb.coop Mon Jan 27 21:49:43 2014 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 27 Jan 2014 14:49:43 -0600 Subject: [M3devel] narrow still failing.. In-Reply-To: <52E13487.9090504@lcwb.coop> References: <20140119213649.403AE1A208F@async.async.caltech.edu> <20140119220123.0D1971A208F@async.async.caltech.edu> <52E0202F.3020307@lcwb.coop> <20140122204749.C724E1A208F@async.async.caltech.edu> <52E13487.9090504@lcwb.coop> Message-ID: <52E6C667.6030708@lcwb.coop> It should be fixed in the head, now. On 01/23/2014 09:25 AM, Rodney M. Bates wrote: > > > On 01/22/2014 02:47 PM, mika at async.caltech.edu wrote: >> Aha so declaring the type as <: ROOT will probably work as a workaround? >> > > I'm not sure whether that would do it or not. In my pared-down case, I had > moved BDD.T and BDD.Root into main, but left the REFANY variables. That made > the failures go away. > > Until it is fixed, I think the best workaround is TYPECASE with a binding: > > TYPECASE x > OF BDD.T(b) => > Format(b), etc. > > This is known to be working now. You've probably thought about this, but > this way requires only a single runtime type test. Either TYPECASE without > the binding of b, or ISTYPE, will require an additional recheck of the same > thing when the narrow, implicit or explicit, happens. I doubt even a > pretty good optimizing compiler could remove this. For ISTYPE, it would have > to know that the runtime routine (e.g. CheckIsType) was a pure function. Pretty > unlikely. Even worse for TYPECASE, where it would have to know a lot about the > relation between CheckIsType and ScanTypecase. > >> Thank you for looking into this. >> >> "Rodney M. Bates" writes: >>> Some more info: >>> >>> The front end is generating incorrect narrow code, only >>> where the type T is an unrevealed <: REFANY. When T and Root >>> are known, it calls RTHooks.CheckIsType for ISTYPE, NARROW, >>> and implicit narrow in an assignment. >>> >>> When opaque, for explicit and implicit narrow, it simply compares >>> the typecode of x to a single typecode (no doubt for T) and demands >>> they be equal. In this case, the actual allocated type of x is Root, >>> a proper subtype of T. >>> >>> For ISTYPE, it does call CheckIsType even when the types are opaque. >>> >>> The fact that it happens only when the types are opaque lends >>> credibility to the fact that this bug has gone unnoticed at least >>> as far back as PM3. >> > > From jay.krell at cornell.edu Tue Jan 28 00:13:46 2014 From: jay.krell at cornell.edu (Jay) Date: Mon, 27 Jan 2014 15:13:46 -0800 Subject: [M3devel] Fwd: Why does std::chrono now() uses slow syscall? References: Message-ID: <9FD55748-9752-4E40-9605-78F98AE62A5D@gmail.com> Hints for how m3core/libm3 should get time efficiently... - Jay Begin forwarded message: > From: Jonathan Wakely > Date: January 26, 2014, 2:33:36 AM PST > To: Keith Erickson > Cc: gcc-help > Subject: Re: Why does std::chrono now() uses slow syscall? > > On 26 January 2014 07:50, Keith Erickson wrote: >> My questions, then, are.... why would I ever want to use the syscall >> version? It seems that configure prefers it, and will use it if it's >> available. > > It prefers to use clock_gettime and only uses the syscall if a > configure test using clock_gettime(CLOCK_MONOTONIC, &tp) fails. > >> But it's stupidly slow, even on the fasted server CPU that >> AMD sells (Opteron 6386 SE at the time of this writing). How do I not >> use this slow method? Do I have to compile gcc specially? What are >> the drawbacks to forcing configure to not set that #define? Is there >> an approved way to tell configure to use a fast time? > > Prior to glibc 2.17 clock_gettime was in librt.so not libc.so, which > is not used automatically by libstdc++. To tell configure to use it > you need to build GCC with --enable-libstdcxx-time=rt, but that means > that libstdc++.so will be linked to librt.so which depends on > libpthread.so which means that the library always thinks it is part of > a multithreaded application and so even single-threaded programs use > atomic operations for internal reference-counting (e.g. in std::string > and std::shared_ptr). That's why we don't use clock_gettime unless > explicitly configured to do so. > > With glibc 2.17 or later clock_gettime will be used automatically, > because the functions were moved out of librt.so. > > On my Fedora 20 system I get > > | #define HAVE_SYS_TIME_H 1 > | #define _GLIBCXX_USE_GETTIMEOFDAY 1 > | #define _GLIBCXX_USE_CLOCK_MONOTONIC 1 > | #define _GLIBCXX_USE_CLOCK_REALTIME 1 > | #define _GLIBCXX_USE_SCHED_YIELD 1 > | #define _GLIBCXX_USE_NANOSLEEP 1 > > which means it doesn't need the syscall version. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Wed Jan 29 00:49:19 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Tue, 28 Jan 2014 15:49:19 -0800 Subject: [M3devel] garbage collector compacting and volatile? In-Reply-To: <4798B41F-079F-463C-A4FB-C36185B29997@cs.purdue.edu> References: , <8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu> <4798B41F-079F-463C-A4FB-C36185B29997@cs.purdue.edu> Message-ID: <20140128234919.0E0C81A208F@async.async.caltech.edu> Sounds like Jay doesn't need to buy a copy of your book :-) Tony Hosking writes: > >--Apple-Mail=_12C5DBD9-678E-4559-B4C2-E4CFDA144CD4 >Content-Transfer-Encoding: quoted-printable >Content-Type: text/plain; > charset=iso-8859-1 > > > >On Jan 27, 2014, at 2:45 AM, Jay K wrote: > >> In summary, shortest: >>=20 >>=20 >> We must be able to read registers (or save them to the stack or = >elsewhere (known thread locals)). =20 >> We must be able to read the stack. =20 >> We need not write registers or stack (from garbage collector). =20 > >Correct. > >> Slight elaboration: >> Copying/compaction only occurs when references are only from globals = >and heap.=20 >> Therefore updates/writes only occur to globals and heap.=20 > >Correct. > >> Is there any desire to do better/different?=20 >> To require the ability to update stack/registers?=20 > >Without control of the register allocator, optimisations, and stack = >layout performed by the compiler (especially back-end) getting fully = >accurate stack maps is very difficult. > >> What does Java typically to?=20 > >Java VMs usually control their own destiny, having full control of JIT = >and stack layout. viz. HotSpot, IBM J9, Jikes RVM. > >> I guess there is the problem that we don't know for certain if values = >in registers and stack are pointers, or just coincident integers, and so = >updating them can be wrong. > >Correct. > >> Whereas in the globals and "reachable thereof" we have precise type = >information for, so we can update those values safely. > >Correct. > >> To do better/different would require more backend cooperation. >> We conservatively treat registers/stack as pointers if they look like = >them, and therefore keep values alive, but not trust them so much as to = >update them. > >Correct. From jay.krell at cornell.edu Wed Jan 29 01:00:20 2014 From: jay.krell at cornell.edu (Jay K) Date: Wed, 29 Jan 2014 00:00:20 +0000 Subject: [M3devel] garbage collector compacting and volatile? In-Reply-To: <20140128234919.0E0C81A208F@async.async.caltech.edu> References: , , <8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu>, , <4798B41F-079F-463C-A4FB-C36185B29997@cs.purdue.edu>, <20140128234919.0E0C81A208F@async.async.caltech.edu> Message-ID: Good point. I have it around somewhere... (I need that automatic resource management in real life..) - Jay > To: hosking at cs.purdue.edu > Date: Tue, 28 Jan 2014 15:49:19 -0800 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] garbage collector compacting and volatile? > > > Sounds like Jay doesn't need to buy a copy of your book :-) > > Tony Hosking writes: > > > >--Apple-Mail=_12C5DBD9-678E-4559-B4C2-E4CFDA144CD4 > >Content-Transfer-Encoding: quoted-printable > >Content-Type: text/plain; > > charset=iso-8859-1 > > > > > > > >On Jan 27, 2014, at 2:45 AM, Jay K wrote: > > > >> In summary, shortest: > >>=20 > >>=20 > >> We must be able to read registers (or save them to the stack or = > >elsewhere (known thread locals)). =20 > >> We must be able to read the stack. =20 > >> We need not write registers or stack (from garbage collector). =20 > > > >Correct. > > > >> Slight elaboration: > >> Copying/compaction only occurs when references are only from globals = > >and heap.=20 > >> Therefore updates/writes only occur to globals and heap.=20 > > > >Correct. > > > >> Is there any desire to do better/different?=20 > >> To require the ability to update stack/registers?=20 > > > >Without control of the register allocator, optimisations, and stack = > >layout performed by the compiler (especially back-end) getting fully = > >accurate stack maps is very difficult. > > > >> What does Java typically to?=20 > > > >Java VMs usually control their own destiny, having full control of JIT = > >and stack layout. viz. HotSpot, IBM J9, Jikes RVM. > > > >> I guess there is the problem that we don't know for certain if values = > >in registers and stack are pointers, or just coincident integers, and so = > >updating them can be wrong. > > > >Correct. > > > >> Whereas in the globals and "reachable thereof" we have precise type = > >information for, so we can update those values safely. > > > >Correct. > > > >> To do better/different would require more backend cooperation. > >> We conservatively treat registers/stack as pointers if they look like = > >them, and therefore keep values alive, but not trust them so much as to = > >update them. > > > >Correct. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Wed Jan 29 01:07:15 2014 From: hosking at cs.purdue.edu (Tony Hosking) Date: Tue, 28 Jan 2014 19:07:15 -0500 Subject: [M3devel] garbage collector compacting and volatile? In-Reply-To: <20140128234919.0E0C81A208F@async.async.caltech.edu> References: , <8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu> <4798B41F-079F-463C-A4FB-C36185B29997@cs.purdue.edu> <20140128234919.0E0C81A208F@async.async.caltech.edu> Message-ID: <38455303-D2F4-4BDD-8E31-D7139D566436@cs.purdue.edu> ;-) I should send him a copy! On Jan 28, 2014, at 6:49 PM, mika at async.caltech.edu wrote: > > Sounds like Jay doesn't need to buy a copy of your book :-) > > Tony Hosking writes: >> >> --Apple-Mail=_12C5DBD9-678E-4559-B4C2-E4CFDA144CD4 >> Content-Transfer-Encoding: quoted-printable >> Content-Type: text/plain; >> charset=iso-8859-1 >> >> >> >> On Jan 27, 2014, at 2:45 AM, Jay K wrote: >> >>> In summary, shortest: >>> =20 >>> =20 >>> We must be able to read registers (or save them to the stack or = >> elsewhere (known thread locals)). =20 >>> We must be able to read the stack. =20 >>> We need not write registers or stack (from garbage collector). =20 >> >> Correct. >> >>> Slight elaboration: >>> Copying/compaction only occurs when references are only from globals = >> and heap.=20 >>> Therefore updates/writes only occur to globals and heap.=20 >> >> Correct. >> >>> Is there any desire to do better/different?=20 >>> To require the ability to update stack/registers?=20 >> >> Without control of the register allocator, optimisations, and stack = >> layout performed by the compiler (especially back-end) getting fully = >> accurate stack maps is very difficult. >> >>> What does Java typically to?=20 >> >> Java VMs usually control their own destiny, having full control of JIT = >> and stack layout. viz. HotSpot, IBM J9, Jikes RVM. >> >>> I guess there is the problem that we don't know for certain if values = >> in registers and stack are pointers, or just coincident integers, and so = >> updating them can be wrong. >> >> Correct. >> >>> Whereas in the globals and "reachable thereof" we have precise type = >> information for, so we can update those values safely. >> >> Correct. >> >>> To do better/different would require more backend cooperation. >>> We conservatively treat registers/stack as pointers if they look like = >> them, and therefore keep values alive, but not trust them so much as to = >> update them. >> >> Correct. From mika at async.caltech.edu Wed Jan 1 11:48:36 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Wed, 01 Jan 2014 02:48:36 -0800 Subject: [M3devel] mklib not compiling Message-ID: <20140101104836.E13C51A208E@async.async.caltech.edu> Hi m3devel, Trying to upgrade my compiler on AMD64_LINUX... === package /home/mika/cm3-cvs-anon/cm3/m3-sys/mklib === +++ cm3 -build -DROOT='/home/mika/cm3-cvs-anon/cm3' $RARGS && cm3 -ship $RARGS -DROOT='/home/mika/cm3-cvs-anon/cm3' +++ --- building in AMD64_LINUX --- ignoring ../src/m3overrides new source -> compiling Main.m3 "../src/Main.m3", line 659: incompatible types (b) "../src/Main.m3", line 660: incompatible types (b) "../src/Main.m3", line 661: incompatible types (b) "../src/Main.m3", line 662: incompatible types (b) "../src/Main.m3", line 663: incompatible types (b) "../src/Main.m3", line 664: incompatible types (b) "../src/Main.m3", line 665: incompatible types (b) 7 errors encountered compilation failed => not building program "mklib" Fatal Error: package build failed I'm at the head... Mika From mika at async.caltech.edu Thu Jan 2 17:34:10 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Thu, 02 Jan 2014 08:34:10 -0800 Subject: [M3devel] building the CVS head---how??? Message-ID: <20140102163410.757DB1A2080@async.async.caltech.edu> So here is the situation. New machine, new problems. AMD64_LINUX. I want to try the latest compiler, because there are issues with what I have. What I have is the following: Critical Mass Modula-3 version 5.8.6 last updated: 2010-04-11 compiled: 2010-07-12 20:10:34 configuration: /home/mika/cm3.amd64-linux/bin/cm3.cfg host: AMD64_LINUX target: AMD64_LINUX I think the boot archives on the CM3 web site are about the same age. I try ./boot1.py AMD64_LINUX and I get "../src/win32/WinSock.i3", line 662: unsupported language or calling convention (PASCAL) "../src/win32/WinSock.i3", line 665: unsupported language or calling convention (PASCAL) "../src/win32/WinSock.i3", line 688: unsupported language or calling convention (PASCAL) "../src/win32/WinSock.i3", line 691: unsupported language or calling convention (PASCAL) "../src/win32/WinSock.i3", line 694: unsupported language or calling convention (PASCAL) "../src/win32/WinSock.i3", line 697: unsupported language or calling convention (PASCAL) 34 errors encountered new exporters -> recompiling WinVer.i3 "../src/win32/WinVer.i3", line 158: unsupported language or calling convention (APIENTRY) "../src/win32/WinVer.i3", line 168: unsupported language or calling convention (APIENTRY) "../src/win32/WinVer.i3", line 180: unsupported language or calling convention (APIENTRY) "../src/win32/WinVer.i3", line 190: unsupported language or calling convention (APIENTRY) "../src/win32/WinVer.i3", line 203: unsupported language or calling convention (APIENTRY) (etc) all right, it's broken, something needs to be fixed. WHAT IS NOT ALL RIGHT: At this point my regular cm3.cfg is messed up. If I had been silly enough to try to bootstrap a new compiler (for a different machine) without first saving my old CM3 installation, I'd be screwed. But from reading the mailing list logs, I am not sure if it is even possible to bootstrap the head from the older installation. How am I supposed to get a new compiler working? Build it on my Raspberry Pi (which has a working, recent CM3 on it)? (After backing up my old installation, of course.) And what's this? Bad python? (119)truffles:~/cm3-cvs-anon/cm3/scripts/python>./boot1.py AMD64_LINUX Traceback (most recent call last): File "./boot1.py", line 5, in import pylib File "/big/home2/mika/2/cm3-cvs/cm3/scripts/python/pylib.py", line 587, in for a in os.popen(CM3 + " -version 2>" + DevNull): TypeError: unsupported operand type(s) for +: 'NoneType' and 'str' (120)truffles:~/cm3-cvs-anon/cm3/scripts/python>python --version Python 2.7.3 Mika Mika From mika at async.caltech.edu Fri Jan 3 00:15:37 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Thu, 02 Jan 2014 15:15:37 -0800 Subject: [M3devel] booting for AMD64_LINUX, continued... Message-ID: <20140102231537.B953A1A2080@async.async.caltech.edu> So I'm trying to get this working by cross-compiling from the ARM_LINUX to AMD64_LINUX by doing ./boot1.py c AMD64_LINUX I get: new source -> compiling PklActionSeq.i3 new source -> compiling ConvertPacking.m3 "../src/pickle/ver2/ConvertPacking.m3", line 327: warning: not used (ReadWC21) "../src/pickle/ver2/ConvertPacking.m3", line 851: warning: not used (WriteWC21) *** *** runtime error: *** <*ASSERT*> failed. *** file "../src/M3C.m3", line 5890 *** Aborted *** execution of [] failed *** 5887 PROCEDURE shift_right(self: T; type: IType) = 5888 (* s1.type := Word.Shift (s1.type, -s0.type); pop *) 5889 BEGIN 5890 <* ASSERT cgtypeIsUnsignedInt[type] *> 5891 shift_left_or_right(self, type, "shift_right", ">>"); 5892 END shift_right; ... Does anyone have a relatively recent working AMD64_LINUX installation? I'd love a copy to try to boot from... Mika From mika at async.caltech.edu Fri Jan 3 06:12:52 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Thu, 02 Jan 2014 21:12:52 -0800 Subject: [M3devel] booting for AMD64_LINUX, continued... In-Reply-To: <20140102174632.3169C41A@resin13.mta.everyone.net> References: <20140102174632.3169C41A@resin13.mta.everyone.net> Message-ID: <20140103051252.88FC61A2080@async.async.caltech.edu> Well I managed to get it to work using the following roundabout procedure: Since the only archives I could find on elegosoft for AMD64_LINUX were ancient, I installed VirtualBox and FreeBSD/amd64 inside that. FreeBSD/amd64 for some reason has very recent archives (of the old "cminstall" kind). Downloaded the FreeBSD archive, cminstalled it, and ran upgrade.py on the latest sources. Had to remove some code from ConvertPacking.m3 because Jay's C-backend can't compile that for some reason. ran boot1.py, which gave me a boot archive. Note---if you've done this with boot1.py "c" before, you have to delete all the derived dirs and start over, or you get a mess. Moved the boot archive to the Linux machine. Resurrected ConvertPacking.m3 Ran boot2.sh (or equiv) until I got everything built. That worked. So... some observations: .. I have a very recent build on ARM_LINUX, but wasn't able to cross-compile. Word size problem? .. I now have three very recent builds: ARM_LINUX, AMD64_FREEBSD, and AMD64_LINUX. What precisely do I have to do to make boot archives out of these so other people might be spared this painful process? Very few archives on the elego site are up to date. Some NT stuff and some FreeBSD stuff. Nothing else that I can see. Mika "Rodney Bates" writes: > > >-Rodney Bates > >--- mika at async.caltech.edu wrote: > >From: mika at async.caltech.edu >To: m3devel at elegosoft.com >Subject: [M3devel] booting for AMD64_LINUX, continued... >Date: Thu, 02 Jan 2014 15:15:37 -0800 > > >So I'm trying to get this working by cross-compiling from the ARM_LINUX to AMD64_LINUX by doing > >./boot1.py c AMD64_LINUX > >I get: > >new source -> compiling PklActionSeq.i3 >new source -> compiling ConvertPacking.m3 >"../src/pickle/ver2/ConvertPacking.m3", line 327: warning: not used (ReadWC21) >"../src/pickle/ver2/ConvertPacking.m3", line 851: warning: not used (WriteWC21) > > >*** >*** runtime error: >*** <*ASSERT*> failed. >*** file "../src/M3C.m3", line 5890 >*** > >Aborted > *** execution of [] failed *** > > > 5887 PROCEDURE shift_right(self: T; type: IType) = > 5888 (* s1.type := Word.Shift (s1.type, -s0.type); pop *) > 5889 BEGIN > 5890 <* ASSERT cgtypeIsUnsignedInt[type] *> > 5891 shift_left_or_right(self, type, "shift_right", ">>"); > 5892 END shift_right; > >... > >Does anyone have a relatively recent working AMD64_LINUX installation? I'd love a copy to try to boot from... > >I have one, but am out of the country right now and have no access to it. >I am waiting as I write to board the first airplane home, so will try to >tar one up for you when I get there. > > > Mika > From mika at async.caltech.edu Fri Jan 3 06:57:18 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Thu, 02 Jan 2014 21:57:18 -0800 Subject: [M3devel] pthreads still broken Message-ID: <20140103055718.B12821A2080@async.async.caltech.edu> On my new AMD64_LINUX build... The thread tester still crashes. First, it prints a couple of extraneous diagnostics. These look like they come from the threading code? Then it segfaults. (141)truffles:~/cm3-cvs-anon/cm3/m3-libs/m3core/tests/thread/AMD64_LINUX>gdb threadtest GNU gdb (GDB) 7.4.1-debian Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/thread/AMD64_LINUX/threadtest...done. (gdb) run Starting program: /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/thread/AMD64_LINUX/threadtest [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Writing file...done Creating read threads...[New Thread 0x2aaaab6f8700 (LWP 23996)] [New Thread 0x2aaaab8f9700 (LWP 23997)] [New Thread 0x2aaaabafa700 (LWP 23998)] done Creating fork threads...[New Thread 0x2aaaabcfb700 (LWP 23999)] [New Thread 0x2aaaabefc700 (LWP 24000)] [New Thread 0x2aaaac0fd700 (LWP 24001)] done Creating alloc threads...[New Thread 0x2aaaac2fe700 (LWP 24002)] [New Thread 0x2aaaac4ff700 (LWP 24003)] [New Thread 0x2aaaac700700 (LWP 24004)] done Creating lock threads...[New Thread 0x2aaaac901700 (LWP 24005)] [New Thread 0x2aaaacb02700 (LWP 24006)] [New Thread 0x2aaaacd03700 (LWP 24007)] done running...printing oldest/median age/newest Program received signal SIG64, Real-time event 64. [Switching to Thread 0x2aaaac901700 (LWP 24005)] 0x00002aaaaaf5b64b in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0 (gdb) handle SIG64 nostop noprint Signal Stop Print Pass to program Description SIG64 No No Yes Real-time event 64 (gdb) cont Continuing. ....pthread_mutex_destroy:EBUSY ...pthread_mutex_destroy:EBUSY Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x2aaaab6f8700 (LWP 23996)] 0x0000000000416de6 in FileRd__Init (M3_DaDE31_rd=, M3_EQj1Bs_h=) at ../src/rw/FileRd.m3:44 44 IF (rd.buff = NIL) THEN (gdb) where #0 0x0000000000416de6 in FileRd__Init (M3_DaDE31_rd=, M3_EQj1Bs_h=) at ../src/rw/FileRd.m3:44 #1 0x0000000000416d3a in FileRd__Open (M3_Bd56fi_p=) at ../src/rw/FileRd.m3:16 #2 0x0000000000405424 in Main__RApply (M3_AP7a1g_cl=) at ../src/Main.m3:182 #3 0x000000000044c23b in ThreadPThread__RunThread (M3_DMxDjQ_me=) at ../src/thread/PTHREAD/ThreadPThread.m3:449 #4 0x000000000044bef8 in ThreadPThread__ThreadBase (M3_AJWxb1_param=) at ../src/thread/PTHREAD/ThreadPThread.m3:422 #5 0x00002aaaaaf56b50 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #6 0x00002aaaab246a7d in clone () from /lib/x86_64-linux-gnu/libc.so.6 #7 0x0000000000000000 in ?? () (gdb) The reason I even ran it was because I found that running the parallel compiler backend was unstable. Do user threads work on AMD64_LINUX? Mika From mika at async.caltech.edu Fri Jan 3 17:13:43 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Fri, 03 Jan 2014 08:13:43 -0800 Subject: [M3devel] thread tester results on ARM_LINUX Message-ID: <20140103161343.F25291A2080@async.async.caltech.edu> (78)raspberrypi:~/cm3-cvs-anon/cm3/m3-libs/m3core/tests/thread>gdb ARM_LINUX/threadtest GNU gdb (GDB) 7.4.1-debian Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "arm-linux-gnueabihf". For bug reporting instructions, please see: ... Reading symbols from /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/thread/ARM_LINUX/threadtest...done. (gdb) handle SIGILL Signal Stop Print Pass to program Description SIGILL Yes Yes Yes Illegal instruction (gdb) handle SIGILL nostop noprint Signal Stop Print Pass to program Description SIGILL No No Yes Illegal instruction (gdb) run Starting program: /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/thread/ARM_LINUX/threadtest [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1". Writing file...done Creating read threads...[New Thread 0x409f4450 (LWP 2941)] [New Thread 0x40bf4450 (LWP 2942)] [New Thread 0x40df4450 (LWP 2943)] done Creating fork threads...[New Thread 0x40ff4450 (LWP 2944)] [New Thread 0x411f4450 (LWP 2945)] [New Thread 0x413f4450 (LWP 2946)] done Creating alloc threads...[New Thread 0x415f4450 (LWP 2947)] [New Thread 0x417f4450 (LWP 2948)] [New Thread 0x419f4450 (LWP 2949)] done Creating lock threads...[New Thread 0x41bf4450 (LWP 2950)] [New Thread 0x41df4450 (LWP 2951)] [New Thread 0x41ff4450 (LWP 2952)] done running...printing oldest/median age/newest Program received signal SIG64, Real-time event 64. [Switching to Thread 0x41ff4450 (LWP 2952)] 0x4028f848 in pthread_cond_timedwait@@GLIBC_2.4 () from /lib/arm-linux-gnueabihf/libpthread.so.0 (gdb) handle SIG64 nostop noprint Signal Stop Print Pass to program Description SIG64 No No Yes Real-time event 64 (gdb) cont Continuing. . *** *** runtime error: *** An array subscript was out of range. *** file "../src/runtime/common/RTCollector.m3", line 418 *** Program received signal SIGABRT, Aborted. [Switching to Thread 0x40ff4450 (LWP 2944)] 0x402cfbfc in raise () from /lib/arm-linux-gnueabihf/libc.so.6 (gdb) where #0 0x402cfbfc in raise () from /lib/arm-linux-gnueabihf/libc.so.6 #1 0x402d397c in abort () from /lib/arm-linux-gnueabihf/libc.so.6 #2 0x00000000 in ?? () (gdb) It runs fine with user threads on AMD64_LINUX now. Haven't tried that on ARM_LINUX but would be surprised if it didn't work. Yes the pthreads seem "more reliable" than they used to be, but I would never use them in any sort of mission-critical application. I use Modula-3 programs for (among other things) financial transactions and real-time motor controllers... Mika From mika at async.caltech.edu Fri Jan 3 18:19:32 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Fri, 03 Jan 2014 09:19:32 -0800 Subject: [M3devel] AMD64_LINUX update... Message-ID: <20140103171932.F28D61A2080@async.async.caltech.edu> Hi again m3devel, With the head and user threads, after bootstrapping via AMD64_FREEBSD, I'm getting everything (apparently) working EXCEPT one thing: stubgen (and my derivative of it, the Scheme-stubgen "sstubgen") goes into an infinite loop, looks like it's happening on the stack because I run out of memory rather quickly, on certain types local to my own system. Has anyone seen this before? I've never seen it on any 32-bit system but I have a vague memory of seeing it on 64-bit systems. Will debug over the weekend... I'd be grateful for any input from the list on the pthreads issues, though. Mika From mika at async.caltech.edu Sat Jan 4 19:55:11 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sat, 04 Jan 2014 10:55:11 -0800 Subject: [M3devel] performance problems in CM3 Text operations on AMD64_LINUX Message-ID: <20140104185511.9000A1A2080@async.async.caltech.edu> I have this program that generates Modula-3 code from XML. It uses Wr.PutText a lot, on small strings. It runs 10 times faster under PM3 on FreeBSD/i386 on a 2 GHz old Pentium III type computer than under CM3 (at head) on Debian/amd64 on a 4 GHz Core i7 4960X. (176)rover:~/t/calarm/fix/gen/src>time ../FreeBSD4/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml 0.419u 0.089s 0:00.70 70.0% 471+27608k 0+546io 0pf+0w (29)truffles:~/t/calarm/fix/gen/src>time ../AMD64_LINUX/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml 1.952u 0.232s 0:05.40 40.3% 0+0k 0+11504io 0pf+0w I believe the issue is that Text.m3 is making a ton of system calls! Here's a typical traceback: (gdb) where #0 0x00002aaaaaacb770 in ?? () #1 0x00002aaaaaacba1b in gettimeofday () #2 0x00002aaaac468f8a in gettimeofday () at ../sysdeps/unix/sysv/linux/x86_64/gettimeofday.S:37 #3 0x000000000048c10c in TimePosix__Now () at ../src/time/POSIX/TimePosixC.c:50 #4 0x000000000048b4d2 in Time__Now () at ../src/time/POSIX/TimePosix.m3:14 #5 0x000000000049cbcf in TextStats__NoteFinished (M3_ESffkp_o=) at ../src/text/TextStats.m3:64 #6 0x00000000004921de in Text__SetChars (M3_CKMnXU_a=, M3_Bd56fi_t=, M3_Cwb5VA_start=) at ../src/text/Text.m3:512 #7 0x0000000000441a26 in UnsafeWr__FastPutText (M3_BxxOH1_wr=, M3_Bd56fi_t=) at ../src/rw/Wr.m3:100 #8 0x000000000044192f in Wr__PutText (M3_BxxOH1_wr=, M3_Bd56fi_t=) at ../src/rw/Wr.m3:89 #9 0x000000000041627f in M3Syntax__TextSetPutList (M3_CT76zs_s=, M3_BxxOH1_wr=, M3_AicXUJ_quoted=) at ../src/M3Syntax.m3:129 #10 0x00000000004169cf in M3Syntax__EnumToText (M3_Bqnwti_e=, M3_ACGgGY_env=) at ../src/M3Syntax.m3:173 #11 0x000000000041791a in M3Syntax__UnstrToText (M3_AIPC2K_u=, M3_ACGgGY_env=) at ../src/M3Syntax.m3:295 #12 0x0000000000414ad4 in M3File__DumpInterface (M3_CjySNq_t=, M3_Bd56fi_dirPath=, M3_AicXUJ_updateMakefile=) at ../src/M3File.m3:78 #13 0x000000000040f3f9 in Fields__Do__MakeFieldsFiles.1150 () at ../AMD64_LINUX/Fields.m3:526 #14 0x000000000040a713 in Fields__Do (M3_DsEykq_xml=, M3_Bd56fi_versionString=, M3_Bd56fi_beginStringV=, M3_D9m5ya_fieldTbl=, M3_Bd56fi_Dir=, M3_Bd56fi_SrcDir=, M3_AZx9O5_binaryFields=, M3_CebKnt_specialFields=, M3_CT76zs_checkedTypes=) at ../AMD64_LINUX/Fields.m3:538 #15 0x0000000000413948 in Main_M3 (M3_AcxOUs_mode=) at ../AMD64_LINUX/Main.m3:431 #16 0x00000000004742ad in RTLinker__RunMainBody (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:408 #17 0x0000000000473638 in RTLinker__AddUnitI (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:115 #18 0x00000000004736cc in RTLinker__AddUnit (M3_DjPxE5_b=) at ../src/runtime/common/RTLinker.m3:124 #19 0x0000000000405df8 in main (argc=8, argv=0x7fffffffe2b8, envp=0x7fffffffe300) at _m3main.c:22 (gdb) I believe all the TextStats operations are killing the performance. Can we remove them? Here is SetChars: PROCEDURE SetChars (VAR a: ARRAY OF CHAR; t: T; start: CARDINAL) = BEGIN TextStats.NoteGround (TextStats.Op.SetChars); TextStats.NoteGround (TextStats.Op.get_chars); t.get_chars (a, start); TextStats.NoteFinished (TextStats.Op.get_chars); TextStats.NoteFinished (TextStats.Op.SetChars); END SetChars; I'm not sure how best to do this. Removing them without killing functionality might not be easy. Looks like a case for conditional compilation. Maybe generate the .m3s from some other source, with and without the TextStats, depending on a setting in m3makefile? Second best would be to have a boolean flag in Text.m3 so the TextStats never get called. Mika From dragisha at m3w.org Sat Jan 4 22:04:22 2014 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 4 Jan 2014 22:04:22 +0100 Subject: [M3devel] AMD64_LINUX update... In-Reply-To: <20140103171932.F28D61A2080@async.async.caltech.edu> References: <20140103171932.F28D61A2080@async.async.caltech.edu> Message-ID: Hi Mika, I have this for LINUXLIBC6, and I am using it for remake of HEAD: cm3-bin-core-LINUXLIBC6-d5.9.0-i686-pc-linux-gnu-2013-11-20-19-55-28.tgz Can you build and share cm3-bin-core for AMD64_LINUX? TIA, dd On 03 Jan 2014, at 18:19, mika at async.caltech.edu wrote: > > Hi again m3devel, > > With the head and user threads, after bootstrapping via AMD64_FREEBSD, > I'm getting everything (apparently) working EXCEPT one thing: > > stubgen (and my derivative of it, the Scheme-stubgen "sstubgen") goes > into an infinite loop, looks like it's happening on the stack because I > run out of memory rather quickly, on certain types local to my own system. > > Has anyone seen this before? I've never seen it on any 32-bit system > but I have a vague memory of seeing it on 64-bit systems. > > Will debug over the weekend... > > I'd be grateful for any input from the list on the pthreads issues, though. > > Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Sat Jan 4 22:12:00 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sat, 04 Jan 2014 13:12:00 -0800 Subject: [M3devel] AMD64_LINUX update... In-Reply-To: References: <20140103171932.F28D61A2080@async.async.caltech.edu> Message-ID: <20140104211200.A80571A2080@async.async.caltech.edu> Sure... what do I do? Is there a script? Or do I tar up a selected set of directories? I can do this for both AMD64_LINUX (user threads) and ARM_LINUX (pthreads) I might still have AMD64_LINUX (pthreads) Also have AMD64_FREEBSD (pthreads) from the head. =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: > >--Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421 >Content-Type: multipart/alternative; > boundary="Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657" > > >--Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657 >Content-Transfer-Encoding: quoted-printable >Content-Type: text/plain; > charset=us-ascii > >Hi Mika, > >I have this for LINUXLIBC6, and I am using it for remake of HEAD: = >cm3-bin-core-LINUXLIBC6-d5.9.0-i686-pc-linux-gnu-2013-11-20-19-55-28.tgz > >Can you build and share cm3-bin-core for AMD64_LINUX? > >TIA, >dd > >On 03 Jan 2014, at 18:19, mika at async.caltech.edu wrote: > >>=20 >> Hi again m3devel, >>=20 >> With the head and user threads, after bootstrapping via AMD64_FREEBSD, >> I'm getting everything (apparently) working EXCEPT one thing: >>=20 >> stubgen (and my derivative of it, the Scheme-stubgen "sstubgen") goes >> into an infinite loop, looks like it's happening on the stack because = >I >> run out of memory rather quickly, on certain types local to my own = >system. >>=20 >> Has anyone seen this before? I've never seen it on any 32-bit system >> but I have a vague memory of seeing it on 64-bit systems. >>=20 >> Will debug over the weekend... >>=20 >> I'd be grateful for any input from the list on the pthreads issues, = >though. >>=20 >> Mika > > >--Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657 >Content-Transfer-Encoding: quoted-printable >Content-Type: text/html; > charset=us-ascii > >charset=3Dus-ascii">-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">Hi = >Mika,

I have this for LINUXLIBC6, and I am using it = >for remake of HEAD: color: rgb(255, 59, 29); font-family: Monaco; font-size: = >13px;">cm3-bin-core-LINUXLIBC6-d5.9.0-i686-pc-linux-gnu-2013-11-20-19-55-2= >8.tgz
color: rgb(255, 59, 29); font-family: Monaco; font-size: = >13px;">
orphans: 2; text-align: -webkit-auto; widows: 2;">Can you build and = >share cm3-bin-core for AMD64_LINUX?
apple-content-edited=3D"true">style=3D"border-collapse: separate; font-family: Candara; = >border-spacing: 0px;">style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: = >Helvetica; font-style: normal; font-variant: normal; font-weight: = >normal; letter-spacing: normal; line-height: normal; orphans: 2; = >text-align: -webkit-auto; text-indent: 0px; text-transform: none; = >white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; ">
break-word; -webkit-nbsp-mode: space; -webkit-line-break: = >after-white-space; ">style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: = >Helvetica; font-style: normal; font-variant: normal; font-weight: = >normal; letter-spacing: normal; line-height: normal; orphans: 2; = >text-align: -webkit-auto; text-indent: 0px; text-transform: none; = >white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; ">
break-word; -webkit-nbsp-mode: space; -webkit-line-break: = >after-white-space; ">
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; = >">TIA,
space; -webkit-line-break: after-white-space; ">dd
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; = >">

type=3D"cite">
Hi again m3devel,

With the head and user = >threads, after bootstrapping via AMD64_FREEBSD,
I'm getting = >everything (apparently) working EXCEPT one thing:

stubgen (and my = >derivative of it, the Scheme-stubgen "sstubgen") goes
into an = >infinite loop, looks like it's happening on the stack because I
run = >out of memory rather quickly, on certain types local to my own = >system.

Has anyone seen this before?  I've never seen it on = >any 32-bit system
but I have a vague memory of seeing it on 64-bit = >systems.

Will debug over the weekend...

I'd be grateful = >for any input from the list on the pthreads issues, though.

= >    Mika

>= > >--Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657-- > >--Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421 >Content-Transfer-Encoding: 7bit >Content-Disposition: attachment; > filename=signature.asc >Content-Type: application/pgp-signature; > name=signature.asc >Content-Description: Message signed with OpenPGP using GPGMail > >-----BEGIN PGP SIGNATURE----- >Comment: GPGTools - http://gpgtools.org > >iQEcBAEBAgAGBQJSyHdWAAoJEJtljYXUJo8x7G4H/i04W2KIer4qcBJO0uVLaI2+ >taHLJ9ewgRyfrd3qfp7YkgmoidxVMy6XMRIuwFpNBjz8YE5YK5TU9Awpmr4UKVvM >zGa9eeWWMg+SX6WmiKnIO4WxiF2oNkp3mybuX4oOjjqUN4nhW0iGtycgIlGkDaR7 >0CL5I4D0P8bSkhunE4iATX9I/k+WVFghZhRWr8BULIHgRlD1xG6NZ0EhCcJk1mY2 >hQBKgJEdpiBUfG84cbu0uE5c3hrposmbjuFFaegG0Q1Pr+zZAnbV5C08RhLawyHn >1VPwWGF1R/d1KgVBr4l19KxNi4aACJtov5ODGXXJbfOFyPi25WF6A4SbjP1aJQg= >=sMKJ >-----END PGP SIGNATURE----- > >--Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421-- From mika at async.caltech.edu Sat Jan 4 22:15:51 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sat, 04 Jan 2014 13:15:51 -0800 Subject: [M3devel] thread tester results on ARM_LINUX In-Reply-To: References: <20140103161343.F25291A2080@async.async.caltech.edu> Message-ID: <20140104211551.54F481A2080@async.async.caltech.edu> Antony Hosking writes: >I wonder of the redzone size for AMD 64 stack frames is not set properly.=20= > Tony, Are you talking about the difference between the ARM and AMD64 results? The AMD64 is also a machine with 12 logical (HT) processors whereas the ARM is a single processor. That could make a difference in how it fails. By the way, would you know: The following output on the amd64: Continuing. ....pthread_mutex_destroy:EBUSY ...pthread_mutex_destroy:EBUSY Is that an assertion failing in the pthreads library? My program isn't printing this stuff: it's coming from ThreadPThread or some such code. Mika > >Sent from my iPhone > >> On Jan 3, 2014, at 10:13 AM, mika at async.caltech.edu wrote: >>=20 >> (78)raspberrypi:~/cm3-cvs-anon/cm3/m3-libs/m3core/tests/thread>gdb ARM_LIN= >UX/threadtest >> GNU gdb (GDB) 7.4.1-debian >> Copyright (C) 2012 Free Software Foundation, Inc. >> License GPLv3+: GNU GPL version 3 or later ml> >> This is free software: you are free to change and redistribute it. >> There is NO WARRANTY, to the extent permitted by law. Type "show copying"= > >> and "show warranty" for details. >> This GDB was configured as "arm-linux-gnueabihf". >> For bug reporting instructions, please see: >> ... >> Reading symbols from /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/th= >read/ARM_LINUX/threadtest...done. >> (gdb) handle SIGILL=20 >> Signal Stop Print Pass to program Description >> SIGILL Yes Yes Yes Illegal instruction >> (gdb) handle SIGILL nostop noprint >> Signal Stop Print Pass to program Description >> SIGILL No No Yes Illegal instruction >> (gdb) run >> Starting program: /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/threa= >d/ARM_LINUX/threadtest=20 >> [Thread debugging using libthread_db enabled] >> Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.= >1". >> Writing file...done >> Creating read threads...[New Thread 0x409f4450 (LWP 2941)] >> [New Thread 0x40bf4450 (LWP 2942)] >> [New Thread 0x40df4450 (LWP 2943)] >> done >> Creating fork threads...[New Thread 0x40ff4450 (LWP 2944)] >> [New Thread 0x411f4450 (LWP 2945)] >> [New Thread 0x413f4450 (LWP 2946)] >> done >> Creating alloc threads...[New Thread 0x415f4450 (LWP 2947)] >> [New Thread 0x417f4450 (LWP 2948)] >> [New Thread 0x419f4450 (LWP 2949)] >> done >> Creating lock threads...[New Thread 0x41bf4450 (LWP 2950)] >> [New Thread 0x41df4450 (LWP 2951)] >> [New Thread 0x41ff4450 (LWP 2952)] >> done >> running...printing oldest/median age/newest >>=20 >> Program received signal SIG64, Real-time event 64. >> [Switching to Thread 0x41ff4450 (LWP 2952)] >> 0x4028f848 in pthread_cond_timedwait@@GLIBC_2.4 () from /lib/arm-linux-gnu= >eabihf/libpthread.so.0 >> (gdb) handle SIG64 nostop noprint >> Signal Stop Print Pass to program Description >> SIG64 No No Yes Real-time event 64 >> (gdb) cont >> Continuing. >> . >>=20 >> *** >> *** runtime error: >> *** An array subscript was out of range. >> *** file "../src/runtime/common/RTCollector.m3", line 418 >> *** >>=20 >>=20 >> Program received signal SIGABRT, Aborted. >> [Switching to Thread 0x40ff4450 (LWP 2944)] >> 0x402cfbfc in raise () from /lib/arm-linux-gnueabihf/libc.so.6 >> (gdb) where >> #0 0x402cfbfc in raise () from /lib/arm-linux-gnueabihf/libc.so.6 >> #1 0x402d397c in abort () from /lib/arm-linux-gnueabihf/libc.so.6 >> #2 0x00000000 in ?? () >> (gdb)=20 >>=20 >> It runs fine with user threads on AMD64_LINUX now. Haven't tried that on >> ARM_LINUX but would be surprised if it didn't work. >>=20 >> Yes the pthreads seem "more reliable" than they used to be, but I would >> never use them in any sort of mission-critical application. I use >> Modula-3 programs for (among other things) financial transactions and >> real-time motor controllers... >>=20 >> Mika From dragisha at m3w.org Sat Jan 4 22:52:32 2014 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 4 Jan 2014 22:52:32 +0100 Subject: [M3devel] AMD64_LINUX update... In-Reply-To: <20140104211200.A80571A2080@async.async.caltech.edu> References: <20140103171932.F28D61A2080@async.async.caltech.edu> <20140104211200.A80571A2080@async.async.caltech.edu> Message-ID: -rwxrwxr-x. 1 root 969 Feb 1 2010 scripts/do-cm3-all.sh* -rwxrwxr-x. 1 root 985 Feb 1 2010 scripts/do-cm3-base.sh* -rwxrwxr-x. 1 root 994 Feb 1 2010 scripts/do-cm3-caltech-parser.sh* -rwxrwxr-x. 1 root 998 Feb 1 2010 scripts/do-cm3-coll.sh* -rwxrwxr-x. 1 root 984 Feb 1 2010 scripts/do-cm3-comm.sh* -rwxrwxr-x. 1 root 985 Feb 1 2010 scripts/do-cm3-core.sh* -rwxrwxr-x. 1 root 1072 Feb 1 2010 scripts/do-cm3-front.sh* -rwxrwxr-x. 1 root 984 Feb 1 2010 scripts/do-cm3-gui.sh* -rwxrwxr-x. 1 root 926 Jun 7 2009 scripts/do-cm3-m3gdb.sh* -rwxrwxr-x. 1 root 984 Feb 1 2010 scripts/do-cm3-min.sh* -rwxrwxr-x. 1 root 969 Nov 11 2010 scripts/do-cm3-std.sh* -rwxrwxr-x. 1 root 938 Feb 6 2013 scripts/do-pkg.sh* there they are. I don?t do this too often, but both do-cm3-core and do-cm3-min will do for what I need. TIA, dd On 04 Jan 2014, at 22:12, mika at async.caltech.edu wrote: > Sure... what do I do? Is there a script? Or do I tar up a selected set of directories? > > I can do this for both AMD64_LINUX (user threads) > and ARM_LINUX (pthreads) > > I might still have AMD64_LINUX (pthreads) > > Also have AMD64_FREEBSD (pthreads) from the head. > > =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: >> >> --Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421 >> Content-Type: multipart/alternative; >> boundary="Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657" >> >> >> --Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657 >> Content-Transfer-Encoding: quoted-printable >> Content-Type: text/plain; >> charset=us-ascii >> >> Hi Mika, >> >> I have this for LINUXLIBC6, and I am using it for remake of HEAD: = >> cm3-bin-core-LINUXLIBC6-d5.9.0-i686-pc-linux-gnu-2013-11-20-19-55-28.tgz >> >> Can you build and share cm3-bin-core for AMD64_LINUX? >> >> TIA, >> dd >> >> On 03 Jan 2014, at 18:19, mika at async.caltech.edu wrote: >> >>> =20 >>> Hi again m3devel, >>> =20 >>> With the head and user threads, after bootstrapping via AMD64_FREEBSD, >>> I'm getting everything (apparently) working EXCEPT one thing: >>> =20 >>> stubgen (and my derivative of it, the Scheme-stubgen "sstubgen") goes >>> into an infinite loop, looks like it's happening on the stack because = >> I >>> run out of memory rather quickly, on certain types local to my own = >> system. >>> =20 >>> Has anyone seen this before? I've never seen it on any 32-bit system >>> but I have a vague memory of seeing it on 64-bit systems. >>> =20 >>> Will debug over the weekend... >>> =20 >>> I'd be grateful for any input from the list on the pthreads issues, = >> though. >>> =20 >>> Mika >> >> >> --Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657 >> Content-Transfer-Encoding: quoted-printable >> Content-Type: text/html; >> charset=us-ascii >> >> > charset=3Dus-ascii">> -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">Hi = >> Mika,

I have this for LINUXLIBC6, and I am using it = >> for remake of HEAD: > color: rgb(255, 59, 29); font-family: Monaco; font-size: = >> 13px;">cm3-bin-core-LINUXLIBC6-d5.9.0-i686-pc-linux-gnu-2013-11-20-19-55-2= >> 8.tgz
> color: rgb(255, 59, 29); font-family: Monaco; font-size: = >> 13px;">
> orphans: 2; text-align: -webkit-auto; widows: 2;">Can you build and = >> share cm3-bin-core for AMD64_LINUX?
> apple-content-edited=3D"true">> style=3D"border-collapse: separate; font-family: Candara; = >> border-spacing: 0px;">> style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: = >> Helvetica; font-style: normal; font-variant: normal; font-weight: = >> normal; letter-spacing: normal; line-height: normal; orphans: 2; = >> text-align: -webkit-auto; text-indent: 0px; text-transform: none; = >> white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; ">
> break-word; -webkit-nbsp-mode: space; -webkit-line-break: = >> after-white-space; ">> style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: = >> Helvetica; font-style: normal; font-variant: normal; font-weight: = >> normal; letter-spacing: normal; line-height: normal; orphans: 2; = >> text-align: -webkit-auto; text-indent: 0px; text-transform: none; = >> white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; ">
> break-word; -webkit-nbsp-mode: space; -webkit-line-break: = >> after-white-space; ">
> -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; = >> ">TIA,
> space; -webkit-line-break: after-white-space; ">dd
> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >> -webkit-line-break: after-white-space; = >> ">

> type=3D"cite">
Hi again m3devel,

With the head and user = >> threads, after bootstrapping via AMD64_FREEBSD,
I'm getting = >> everything (apparently) working EXCEPT one thing:

stubgen (and my = >> derivative of it, the Scheme-stubgen "sstubgen") goes
into an = >> infinite loop, looks like it's happening on the stack because I
run = >> out of memory rather quickly, on certain types local to my own = >> system.

Has anyone seen this before?  I've never seen it on = >> any 32-bit system
but I have a vague memory of seeing it on 64-bit = >> systems.

Will debug over the weekend...

I'd be grateful = >> for any input from the list on the pthreads issues, though.

= >>     Mika

>> = >> >> --Apple-Mail=_63AAE786-89CA-4D66-AB37-231EC0069657-- >> >> --Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421 >> Content-Transfer-Encoding: 7bit >> Content-Disposition: attachment; >> filename=signature.asc >> Content-Type: application/pgp-signature; >> name=signature.asc >> Content-Description: Message signed with OpenPGP using GPGMail >> >> -----BEGIN PGP SIGNATURE----- >> Comment: GPGTools - http://gpgtools.org >> >> iQEcBAEBAgAGBQJSyHdWAAoJEJtljYXUJo8x7G4H/i04W2KIer4qcBJO0uVLaI2+ >> taHLJ9ewgRyfrd3qfp7YkgmoidxVMy6XMRIuwFpNBjz8YE5YK5TU9Awpmr4UKVvM >> zGa9eeWWMg+SX6WmiKnIO4WxiF2oNkp3mybuX4oOjjqUN4nhW0iGtycgIlGkDaR7 >> 0CL5I4D0P8bSkhunE4iATX9I/k+WVFghZhRWr8BULIHgRlD1xG6NZ0EhCcJk1mY2 >> hQBKgJEdpiBUfG84cbu0uE5c3hrposmbjuFFaegG0Q1Pr+zZAnbV5C08RhLawyHn >> 1VPwWGF1R/d1KgVBr4l19KxNi4aACJtov5ODGXXJbfOFyPi25WF6A4SbjP1aJQg= >> =sMKJ >> -----END PGP SIGNATURE----- >> >> --Apple-Mail=_382B74FF-31DD-41E6-84F6-E3B9FE47E421-- -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Sat Jan 4 22:58:53 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sat, 04 Jan 2014 13:58:53 -0800 Subject: [M3devel] AMD64_LINUX update... In-Reply-To: References: <20140103171932.F28D61A2080@async.async.caltech.edu> <20140104211200.A80571A2080@async.async.caltech.edu> Message-ID: <20140104215853.09F0D1A2080@async.async.caltech.edu> Hi dd, I already have everything built. What I'm asking is how I go from what I have to a tar file. What is included? Is there a script for that? I can just give you the full binary directory or the full cm3 system with sources, but I suspect there is a reduced set you would prefer? BTW (not sure if this is relevant) but Jay's ./boot1.py and ./boot2.sh scripts in python work fine. If you already have cm3 from the head it ought to be possible to cross-build. But I did have problems going from 32 to 64 bits... Mika =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: > >--Apple-Mail=_F0511A87-0318-4B01-8056-ED3EF8213304 >Content-Type: multipart/alternative; > boundary="Apple-Mail=_67A3EF09-2BA4-4414-BDCB-1E89671A1F12" > > >--Apple-Mail=_67A3EF09-2BA4-4414-BDCB-1E89671A1F12 >Content-Transfer-Encoding: quoted-printable >Content-Type: text/plain; > charset=windows-1252 > >-rwxrwxr-x. 1 root 969 Feb 1 2010 scripts/do-cm3-all.sh* >-rwxrwxr-x. 1 root 985 Feb 1 2010 scripts/do-cm3-base.sh* >-rwxrwxr-x. 1 root 994 Feb 1 2010 scripts/do-cm3-caltech-parser.sh* >-rwxrwxr-x. 1 root 998 Feb 1 2010 scripts/do-cm3-coll.sh* >-rwxrwxr-x. 1 root 984 Feb 1 2010 scripts/do-cm3-comm.sh* >-rwxrwxr-x. 1 root 985 Feb 1 2010 scripts/do-cm3-core.sh* >-rwxrwxr-x. 1 root 1072 Feb 1 2010 scripts/do-cm3-front.sh* >-rwxrwxr-x. 1 root 984 Feb 1 2010 scripts/do-cm3-gui.sh* >-rwxrwxr-x. 1 root 926 Jun 7 2009 scripts/do-cm3-m3gdb.sh* >-rwxrwxr-x. 1 root 984 Feb 1 2010 scripts/do-cm3-min.sh* >-rwxrwxr-x. 1 root 969 Nov 11 2010 scripts/do-cm3-std.sh* >-rwxrwxr-x. 1 root 938 Feb 6 2013 scripts/do-pkg.sh* > >there they are. I don=92t do this too often, but both do-cm3-core and = >do-cm3-min will do for what I need. > >TIA, >dd From dragisha at m3w.org Sat Jan 4 23:10:04 2014 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 4 Jan 2014 23:10:04 +0100 Subject: [M3devel] AMD64_LINUX update... In-Reply-To: <20140104215853.09F0D1A2080@async.async.caltech.edu> References: <20140103171932.F28D61A2080@async.async.caltech.edu> <20140104211200.A80571A2080@async.async.caltech.edu> <20140104215853.09F0D1A2080@async.async.caltech.edu> Message-ID: <7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org> do-cm3-bin will build a tar.gz, I think. And it is not too big. I did not try boot*.py as this box I would like to build for is remote and all? I have src.rpm and I only need this cm3-min? to make it work and build itself on remote box. TIA, dd On 04 Jan 2014, at 22:58, mika at async.caltech.edu wrote: > Hi dd, > > I already have everything built. > > What I'm asking is how I go from what I have to a tar file. What is > included? Is there a script for that? > > I can just give you the full binary directory or the full cm3 system with > sources, but I suspect there is a reduced set you would prefer? > > BTW (not sure if this is relevant) but Jay's ./boot1.py and ./boot2.sh > scripts in python work fine. If you already have cm3 from the head > it ought to be possible to cross-build. But I did have problems going from > 32 to 64 bits... > > Mika > > =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: >> >> --Apple-Mail=_F0511A87-0318-4B01-8056-ED3EF8213304 >> Content-Type: multipart/alternative; >> boundary="Apple-Mail=_67A3EF09-2BA4-4414-BDCB-1E89671A1F12" >> >> >> --Apple-Mail=_67A3EF09-2BA4-4414-BDCB-1E89671A1F12 >> Content-Transfer-Encoding: quoted-printable >> Content-Type: text/plain; >> charset=windows-1252 >> >> -rwxrwxr-x. 1 root 969 Feb 1 2010 scripts/do-cm3-all.sh* >> -rwxrwxr-x. 1 root 985 Feb 1 2010 scripts/do-cm3-base.sh* >> -rwxrwxr-x. 1 root 994 Feb 1 2010 scripts/do-cm3-caltech-parser.sh* >> -rwxrwxr-x. 1 root 998 Feb 1 2010 scripts/do-cm3-coll.sh* >> -rwxrwxr-x. 1 root 984 Feb 1 2010 scripts/do-cm3-comm.sh* >> -rwxrwxr-x. 1 root 985 Feb 1 2010 scripts/do-cm3-core.sh* >> -rwxrwxr-x. 1 root 1072 Feb 1 2010 scripts/do-cm3-front.sh* >> -rwxrwxr-x. 1 root 984 Feb 1 2010 scripts/do-cm3-gui.sh* >> -rwxrwxr-x. 1 root 926 Jun 7 2009 scripts/do-cm3-m3gdb.sh* >> -rwxrwxr-x. 1 root 984 Feb 1 2010 scripts/do-cm3-min.sh* >> -rwxrwxr-x. 1 root 969 Nov 11 2010 scripts/do-cm3-std.sh* >> -rwxrwxr-x. 1 root 938 Feb 6 2013 scripts/do-pkg.sh* >> >> there they are. I don=92t do this too often, but both do-cm3-core and = >> do-cm3-min will do for what I need. >> >> TIA, >> dd -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Sat Jan 4 23:23:21 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sat, 04 Jan 2014 14:23:21 -0800 Subject: [M3devel] can't write to repository Message-ID: <20140104222321.B53051A2080@async.async.caltech.edu> Hi m3devel, I can't seem to write to the CVS repository. I get: (62)rover:~/cm3-writable/cm3/scripts>cvs up -d . mika at birch.elegosoft.com's password: Has my account been removed? I don't think I've changed keys. In any case, could someone please commit the following change: ~/cm3-cvs-anon/cm3/scripts>cvs diff sysinfo.sh Index: sysinfo.sh =================================================================== RCS file: /usr/cvs/cm3/scripts/sysinfo.sh,v retrieving revision 1.109 diff -r1.109 sysinfo.sh 273a274 > arm*) CM3_TARGET=ARM_LINUX;; Mika From mika at async.caltech.edu Sat Jan 4 23:48:13 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sat, 04 Jan 2014 14:48:13 -0800 Subject: [M3devel] AMD64_LINUX update... In-Reply-To: <7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org> References: <20140103171932.F28D61A2080@async.async.caltech.edu> <20140104211200.A80571A2080@async.async.caltech.edu> <20140104215853.09F0D1A2080@async.async.caltech.edu> <7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org> Message-ID: <20140104224813.34D171A2080@async.async.caltech.edu> I think the script you meant was make-bin-dist-min.sh The results are at: http://rover.gcapltd.com/~mika/cm3-bin-core-AMD64_LINUX-d5.9.0-x86_64-unknown-linux-gnu-2014-01-04-22-18-35.tgz Please upload this to the CM3 website if it looks OK to you and you know how to upload it. It's set up to use user threads. Mika From dragisha at m3w.org Sat Jan 4 23:57:55 2014 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 4 Jan 2014 23:57:55 +0100 Subject: [M3devel] AMD64_LINUX update... In-Reply-To: <20140104224813.34D171A2080@async.async.caltech.edu> References: <20140103171932.F28D61A2080@async.async.caltech.edu> <20140104211200.A80571A2080@async.async.caltech.edu> <20140104215853.09F0D1A2080@async.async.caltech.edu> <7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org> <20140104224813.34D171A2080@async.async.caltech.edu> Message-ID: <833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org> I am using it to bootstrap my rpm, did you try copying to: /var/www/modula3.elegosoft.com/cm3/uploaded-archives/ on birch? I will make my rpm tomorrow, and try uploading all of it then. Thanks, dd On 04 Jan 2014, at 23:48, mika at async.caltech.edu wrote: > > I think the script you meant was make-bin-dist-min.sh > > The results are at: > > http://rover.gcapltd.com/~mika/cm3-bin-core-AMD64_LINUX-d5.9.0-x86_64-unknown-linux-gnu-2014-01-04-22-18-35.tgz > > Please upload this to the CM3 website if it looks OK to you and you know how to upload it. > > It's set up to use user threads. > > Mika > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Sat Jan 4 23:59:24 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sat, 04 Jan 2014 14:59:24 -0800 Subject: [M3devel] AMD64_LINUX update... In-Reply-To: <833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org> References: <20140103171932.F28D61A2080@async.async.caltech.edu> <20140104211200.A80571A2080@async.async.caltech.edu> <20140104215853.09F0D1A2080@async.async.caltech.edu> <7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org> <20140104224813.34D171A2080@async.async.caltech.edu> <833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org> Message-ID: <20140104225924.858A21A2080@async.async.caltech.edu> >I am using it to bootstrap my rpm, did you try copying to: = >/var/www/modula3.elegosoft.com/cm3/uploaded-archives/ on birch? Per my email of a few minutes ago, I am unable to get birch to accept my login credentials... From dragisha at m3w.org Sun Jan 5 01:44:17 2014 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 5 Jan 2014 01:44:17 +0100 Subject: [M3devel] AMD64_LINUX update... In-Reply-To: <20140104225924.858A21A2080@async.async.caltech.edu> References: <20140103171932.F28D61A2080@async.async.caltech.edu> <20140104211200.A80571A2080@async.async.caltech.edu> <20140104215853.09F0D1A2080@async.async.caltech.edu> <7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org> <20140104224813.34D171A2080@async.async.caltech.edu> <833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org> <20140104225924.858A21A2080@async.async.caltech.edu> Message-ID: <9CF45913-50ED-45F4-9774-0D0152016105@m3w.org> Copied, and did ?a sensible thing?, ran update script? It did not create as complete index.html as it was before. I will look into it tomorrow, it is too late for me now to believe my further fixing would not break more things :). On 04 Jan 2014, at 23:59, mika at async.caltech.edu wrote: >> I am using it to bootstrap my rpm, did you try copying to: = >> /var/www/modula3.elegosoft.com/cm3/uploaded-archives/ on birch? > > Per my email of a few minutes ago, I am unable to get birch to accept > my login credentials... -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Sun Jan 5 03:50:44 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sat, 04 Jan 2014 18:50:44 -0800 Subject: [M3devel] AMD64_LINUX update... In-Reply-To: <9CF45913-50ED-45F4-9774-0D0152016105@m3w.org> References: <20140103171932.F28D61A2080@async.async.caltech.edu> <20140104211200.A80571A2080@async.async.caltech.edu> <20140104215853.09F0D1A2080@async.async.caltech.edu> <7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org> <20140104224813.34D171A2080@async.async.caltech.edu> <833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org> <20140104225924.858A21A2080@async.async.caltech.edu> <9CF45913-50ED-45F4-9774-0D0152016105@m3w.org> Message-ID: <20140105025044.9FC551A2080@async.async.caltech.edu> Another favor? Could you upload to CM3 the file at http://rover.gcapltd.com/~mika/cm3-bin-core-ARM_LINUX-d5.9.0-armv6l-unknown-linux-gnueabi-2014-01-04-22-23-51.tgz If there is a "notes" section, you can note that the distribution works on 1. Raspberry Pi running Raspbian Wheezy Linux raspberrypi 3.6.11+ #538 PREEMPT Fri Aug 30 20:42:08 BST 2013 armv6l 2. BeagleBone Black running (BeagleBoard)Ubuntu 13.10 (GNU/Linux 3.8.13-bone28 armv7l) Linux arm 3.8.13-bone28 #1 SMP Fri Sep 13 01:11:14 UTC 2013 armv7l armv7l armv7l GNU/Linux (Anyone reading this mail who wants a copy can of course also use the link to rover above.) Mika From dragisha at m3w.org Sun Jan 5 21:15:23 2014 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 5 Jan 2014 21:15:23 +0100 Subject: [M3devel] AMD64_LINUX update... In-Reply-To: <20140105025044.9FC551A2080@async.async.caltech.edu> References: <20140103171932.F28D61A2080@async.async.caltech.edu> <20140104211200.A80571A2080@async.async.caltech.edu> <20140104215853.09F0D1A2080@async.async.caltech.edu> <7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org> <20140104224813.34D171A2080@async.async.caltech.edu> <833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org> <20140104225924.858A21A2080@async.async.caltech.edu> <9CF45913-50ED-45F4-9774-0D0152016105@m3w.org> <20140105025044.9FC551A2080@async.async.caltech.edu> Message-ID: <6F081BEB-ED1C-481F-9389-C36546AAF2D6@m3w.org> Also on http://opencm3.net/snaps/ On 05 Jan 2014, at 03:50, mika at async.caltech.edu wrote: > http://rover.gcapltd.com/~mika/cm3-bin-core-ARM_LINUX-d5.9.0-armv6l-unknown-linux-gnueabi-2014-01-04-22-23-51.tgz -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From dragisha at m3w.org Mon Jan 6 06:02:36 2014 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Mon, 6 Jan 2014 06:02:36 +0100 Subject: [M3devel] AMD64_LINUX update... In-Reply-To: <6F081BEB-ED1C-481F-9389-C36546AAF2D6@m3w.org> References: <20140103171932.F28D61A2080@async.async.caltech.edu> <20140104211200.A80571A2080@async.async.caltech.edu> <20140104215853.09F0D1A2080@async.async.caltech.edu> <7E36BE40-8FB9-4AAB-92BF-2E65F108F9C1@m3w.org> <20140104224813.34D171A2080@async.async.caltech.edu> <833E7B31-C8A4-4127-92C5-05A762E4C778@m3w.org> <20140104225924.858A21A2080@async.async.caltech.edu> <9CF45913-50ED-45F4-9774-0D0152016105@m3w.org> <20140105025044.9FC551A2080@async.async.caltech.edu> <6F081BEB-ED1C-481F-9389-C36546AAF2D6@m3w.org> Message-ID: <9C223752-0979-4BC0-84C7-C8F83B9BF01B@m3w.org> There is hourly run script for user m3 which updates index file in this folder. It keeps deleteng your AMD_LINUX snapshot, apparently after indexing it. ARM_LINUX one is indexed and left in peace. I will look into this issue in few days. dd On 05 Jan 2014, at 21:15, Dragi?a Duri? wrote: > Also on http://opencm3.net/snaps/ > > > On 05 Jan 2014, at 03:50, mika at async.caltech.edu wrote: > >> http://rover.gcapltd.com/~mika/cm3-bin-core-ARM_LINUX-d5.9.0-armv6l-unknown-linux-gnueabi-2014-01-04-22-23-51.tgz > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From rodney_bates at lcwb.coop Mon Jan 6 20:41:53 2014 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 06 Jan 2014 13:41:53 -0600 Subject: [M3devel] performance problems in CM3 Text operations on AMD64_LINUX In-Reply-To: <20140104185511.9000A1A2080@async.async.caltech.edu> References: <20140104185511.9000A1A2080@async.async.caltech.edu> Message-ID: <52CB0701.9030307@lcwb.coop> On 01/04/2014 12:55 PM, mika at async.caltech.edu wrote: > I have this program that generates Modula-3 code from XML. > > It uses Wr.PutText a lot, on small strings. > > It runs 10 times faster under PM3 on FreeBSD/i386 on a 2 GHz old Pentium > III type computer than under CM3 (at head) on Debian/amd64 on a 4 GHz > Core i7 4960X. > > (176)rover:~/t/calarm/fix/gen/src>time ../FreeBSD4/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml > 0.419u 0.089s 0:00.70 70.0% 471+27608k 0+546io 0pf+0w > > (29)truffles:~/t/calarm/fix/gen/src>time ../AMD64_LINUX/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml > 1.952u 0.232s 0:05.40 40.3% 0+0k 0+11504io 0pf+0w > > I believe the issue is that Text.m3 is making a ton of system calls! > > Here's a typical traceback: > > (gdb) where > #0 0x00002aaaaaacb770 in ?? () > #1 0x00002aaaaaacba1b in gettimeofday () > #2 0x00002aaaac468f8a in gettimeofday () at ../sysdeps/unix/sysv/linux/x86_64/gettimeofday.S:37 > #3 0x000000000048c10c in TimePosix__Now () at ../src/time/POSIX/TimePosixC.c:50 > #4 0x000000000048b4d2 in Time__Now () at ../src/time/POSIX/TimePosix.m3:14 > #5 0x000000000049cbcf in TextStats__NoteFinished (M3_ESffkp_o=) at ../src/text/TextStats.m3:64 > #6 0x00000000004921de in Text__SetChars (M3_CKMnXU_a=, M3_Bd56fi_t=, > M3_Cwb5VA_start=) at ../src/text/Text.m3:512 > #7 0x0000000000441a26 in UnsafeWr__FastPutText (M3_BxxOH1_wr=, M3_Bd56fi_t=) at ../src/rw/Wr.m3:100 > #8 0x000000000044192f in Wr__PutText (M3_BxxOH1_wr=, M3_Bd56fi_t=) at ../src/rw/Wr.m3:89 > #9 0x000000000041627f in M3Syntax__TextSetPutList (M3_CT76zs_s=, M3_BxxOH1_wr=, > M3_AicXUJ_quoted=) at ../src/M3Syntax.m3:129 > #10 0x00000000004169cf in M3Syntax__EnumToText (M3_Bqnwti_e=, M3_ACGgGY_env=) at ../src/M3Syntax.m3:173 > #11 0x000000000041791a in M3Syntax__UnstrToText (M3_AIPC2K_u=, M3_ACGgGY_env=) at ../src/M3Syntax.m3:295 > #12 0x0000000000414ad4 in M3File__DumpInterface (M3_CjySNq_t=, M3_Bd56fi_dirPath=, > M3_AicXUJ_updateMakefile=) at ../src/M3File.m3:78 > #13 0x000000000040f3f9 in Fields__Do__MakeFieldsFiles.1150 () at ../AMD64_LINUX/Fields.m3:526 > #14 0x000000000040a713 in Fields__Do (M3_DsEykq_xml=, M3_Bd56fi_versionString=, > M3_Bd56fi_beginStringV=, M3_D9m5ya_fieldTbl=, M3_Bd56fi_Dir=, > M3_Bd56fi_SrcDir=, M3_AZx9O5_binaryFields=, M3_CebKnt_specialFields=, > M3_CT76zs_checkedTypes=) at ../AMD64_LINUX/Fields.m3:538 > #15 0x0000000000413948 in Main_M3 (M3_AcxOUs_mode=) at ../AMD64_LINUX/Main.m3:431 > #16 0x00000000004742ad in RTLinker__RunMainBody (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:408 > #17 0x0000000000473638 in RTLinker__AddUnitI (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:115 > #18 0x00000000004736cc in RTLinker__AddUnit (M3_DjPxE5_b=) at ../src/runtime/common/RTLinker.m3:124 > #19 0x0000000000405df8 in main (argc=8, argv=0x7fffffffe2b8, envp=0x7fffffffe300) at _m3main.c:22 > (gdb) > > I believe all the TextStats operations are killing the performance. Can we remove them? > > Here is SetChars: > > PROCEDURE SetChars (VAR a: ARRAY OF CHAR; t: T; start: CARDINAL) = > BEGIN > TextStats.NoteGround (TextStats.Op.SetChars); > TextStats.NoteGround (TextStats.Op.get_chars); > t.get_chars (a, start); > TextStats.NoteFinished (TextStats.Op.get_chars); > TextStats.NoteFinished (TextStats.Op.SetChars); > END SetChars; > > I'm not sure how best to do this. Removing them without killing functionality might not be > easy. Looks like a case for conditional compilation. > > Maybe generate the .m3s from some other source, with and without the TextStats, depending on > a setting in m3makefile? > > Second best would be to have a boolean flag in Text.m3 so the TextStats never get called. Try the fix I just checked in to TextClass.m3. It was already set up with such a boolean flag (TextClass.CollectStats, set FALSE), but in one of the instrumentation procedures, two system calls were outside the test thereof. Hopefully, this will give a good bit of relief for now, with a simple fix. > > Mika > > From rodney_bates at lcwb.coop Thu Jan 9 22:01:55 2014 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 09 Jan 2014 15:01:55 -0600 Subject: [M3devel] performance problems in CM3 Text operations on AMD64_LINUX In-Reply-To: <20140104185511.9000A1A2080@async.async.caltech.edu> References: <20140104185511.9000A1A2080@async.async.caltech.edu> Message-ID: <52CF0E43.2030701@lcwb.coop> On 01/04/2014 12:55 PM, mika at async.caltech.edu wrote: > I have this program that generates Modula-3 code from XML. > > It uses Wr.PutText a lot, on small strings. > > It runs 10 times faster under PM3 on FreeBSD/i386 on a 2 GHz old Pentium > III type computer than under CM3 (at head) on Debian/amd64 on a 4 GHz > Core i7 4960X. > > (176)rover:~/t/calarm/fix/gen/src>time ../FreeBSD4/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml > 0.419u 0.089s 0:00.70 70.0% 471+27608k 0+546io 0pf+0w > > (29)truffles:~/t/calarm/fix/gen/src>time ../AMD64_LINUX/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml > 1.952u 0.232s 0:05.40 40.3% 0+0k 0+11504io 0pf+0w > > I believe the issue is that Text.m3 is making a ton of system calls! > > Here's a typical traceback: > > (gdb) where > #0 0x00002aaaaaacb770 in ?? () > #1 0x00002aaaaaacba1b in gettimeofday () > #2 0x00002aaaac468f8a in gettimeofday () at ../sysdeps/unix/sysv/linux/x86_64/gettimeofday.S:37 > #3 0x000000000048c10c in TimePosix__Now () at ../src/time/POSIX/TimePosixC.c:50 > #4 0x000000000048b4d2 in Time__Now () at ../src/time/POSIX/TimePosix.m3:14 > #5 0x000000000049cbcf in TextStats__NoteFinished (M3_ESffkp_o=) at ../src/text/TextStats.m3:64 > #6 0x00000000004921de in Text__SetChars (M3_CKMnXU_a=, M3_Bd56fi_t=, > M3_Cwb5VA_start=) at ../src/text/Text.m3:512 > #7 0x0000000000441a26 in UnsafeWr__FastPutText (M3_BxxOH1_wr=, M3_Bd56fi_t=) at ../src/rw/Wr.m3:100 > #8 0x000000000044192f in Wr__PutText (M3_BxxOH1_wr=, M3_Bd56fi_t=) at ../src/rw/Wr.m3:89 > #9 0x000000000041627f in M3Syntax__TextSetPutList (M3_CT76zs_s=, M3_BxxOH1_wr=, > M3_AicXUJ_quoted=) at ../src/M3Syntax.m3:129 > #10 0x00000000004169cf in M3Syntax__EnumToText (M3_Bqnwti_e=, M3_ACGgGY_env=) at ../src/M3Syntax.m3:173 > #11 0x000000000041791a in M3Syntax__UnstrToText (M3_AIPC2K_u=, M3_ACGgGY_env=) at ../src/M3Syntax.m3:295 > #12 0x0000000000414ad4 in M3File__DumpInterface (M3_CjySNq_t=, M3_Bd56fi_dirPath=, > M3_AicXUJ_updateMakefile=) at ../src/M3File.m3:78 > #13 0x000000000040f3f9 in Fields__Do__MakeFieldsFiles.1150 () at ../AMD64_LINUX/Fields.m3:526 > #14 0x000000000040a713 in Fields__Do (M3_DsEykq_xml=, M3_Bd56fi_versionString=, > M3_Bd56fi_beginStringV=, M3_D9m5ya_fieldTbl=, M3_Bd56fi_Dir=, > M3_Bd56fi_SrcDir=, M3_AZx9O5_binaryFields=, M3_CebKnt_specialFields=, > M3_CT76zs_checkedTypes=) at ../AMD64_LINUX/Fields.m3:538 > #15 0x0000000000413948 in Main_M3 (M3_AcxOUs_mode=) at ../AMD64_LINUX/Main.m3:431 > #16 0x00000000004742ad in RTLinker__RunMainBody (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:408 > #17 0x0000000000473638 in RTLinker__AddUnitI (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:115 > #18 0x00000000004736cc in RTLinker__AddUnit (M3_DjPxE5_b=) at ../src/runtime/common/RTLinker.m3:124 > #19 0x0000000000405df8 in main (argc=8, argv=0x7fffffffe2b8, envp=0x7fffffffe300) at _m3main.c:22 > (gdb) > > I believe all the TextStats operations are killing the performance. Can we remove them? > > Here is SetChars: > > PROCEDURE SetChars (VAR a: ARRAY OF CHAR; t: T; start: CARDINAL) = > BEGIN > TextStats.NoteGround (TextStats.Op.SetChars); > TextStats.NoteGround (TextStats.Op.get_chars); > t.get_chars (a, start); > TextStats.NoteFinished (TextStats.Op.get_chars); > TextStats.NoteFinished (TextStats.Op.SetChars); > END SetChars; > > I'm not sure how best to do this. Removing them without killing functionality might not be > easy. Looks like a case for conditional compilation. Now the instrumentation calls are commented out with (*47 ... 74*), which could be reinstated by string search & replace "(*47" -> "(*47*)", etc., and removed again similarly. These are only needed for performance study of the algorithms, which likely won't be done often. > > Maybe generate the .m3s from some other source, with and without the TextStats, depending on > a setting in m3makefile? > > Second best would be to have a boolean flag in Text.m3 so the TextStats never get called. > > Mika > > From jay.krell at cornell.edu Fri Jan 10 09:22:25 2014 From: jay.krell at cornell.edu (Jay K) Date: Fri, 10 Jan 2014 08:22:25 +0000 Subject: [M3devel] performance problems in CM3 Text operations on AMD64_LINUX In-Reply-To: <52CF0E43.2030701@lcwb.coop> References: <20140104185511.9000A1A2080@async.async.caltech.edu>, <52CF0E43.2030701@lcwb.coop> Message-ID: Ok either, way, but here is another point. In some of our code we do: CONST DEBUG = FALSE; IF FALSE THEN xxEND; It will tend to be optimized.Furthermore, syscalls -- even if not optimized,IF FALSE is going to be way faster than a syscall. Also, gettimeofday shouldn't necessarily be a syscall. But/and there are/should-be faster ways of getting time for benchmarking. On Windows, GetTickCount and GetTickCount64 just read a value out of memory.Even GetSystemTimeAsFileTime is just a read of a 64bit value from memory.(The memory is read only, at the same address in all processes, and updated bythe kernel.)x86 has rdtsc, a 64bit hardware cycle counter readable in one instruction. We should consider researching analogs to rdtsc across mips/powerpc/sparc/arm andprovide something in m3core. I think.Add it to the backend interface likely, for inlining.or maybe it is worth a function call, then just use #ifdefed C. Maybe Posix provides something that tends to be like this? Mika, how did you find this?Control-c every so often and look at callstack?Or a lot of stepping? - Jay : Thu, 9 Jan 2014 15:01:55 -0600 > From: rodney_bates at lcwb.coop > To: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] performance problems in CM3 Text operations on AMD64_LINUX > > > > On 01/04/2014 12:55 PM, mika at async.caltech.edu wrote: > > I have this program that generates Modula-3 code from XML. > > > > It uses Wr.PutText a lot, on small strings. > > > > It runs 10 times faster under PM3 on FreeBSD/i386 on a 2 GHz old Pentium > > III type computer than under CM3 (at head) on Debian/amd64 on a 4 GHz > > Core i7 4960X. > > > > (176)rover:~/t/calarm/fix/gen/src>time ../FreeBSD4/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml > > 0.419u 0.089s 0:00.70 70.0% 471+27608k 0+546io 0pf+0w > > > > (29)truffles:~/t/calarm/fix/gen/src>time ../AMD64_LINUX/genfix -o ../../fix42/src -T ../templates -L fix42 FIX42.xml > > 1.952u 0.232s 0:05.40 40.3% 0+0k 0+11504io 0pf+0w > > > > I believe the issue is that Text.m3 is making a ton of system calls! > > > > Here's a typical traceback: > > > > (gdb) where > > #0 0x00002aaaaaacb770 in ?? () > > #1 0x00002aaaaaacba1b in gettimeofday () > > #2 0x00002aaaac468f8a in gettimeofday () at ../sysdeps/unix/sysv/linux/x86_64/gettimeofday.S:37 > > #3 0x000000000048c10c in TimePosix__Now () at ../src/time/POSIX/TimePosixC.c:50 > > #4 0x000000000048b4d2 in Time__Now () at ../src/time/POSIX/TimePosix.m3:14 > > #5 0x000000000049cbcf in TextStats__NoteFinished (M3_ESffkp_o=) at ../src/text/TextStats.m3:64 > > #6 0x00000000004921de in Text__SetChars (M3_CKMnXU_a=, M3_Bd56fi_t=, > > M3_Cwb5VA_start=) at ../src/text/Text.m3:512 > > #7 0x0000000000441a26 in UnsafeWr__FastPutText (M3_BxxOH1_wr=, M3_Bd56fi_t=) at ../src/rw/Wr.m3:100 > > #8 0x000000000044192f in Wr__PutText (M3_BxxOH1_wr=, M3_Bd56fi_t=) at ../src/rw/Wr.m3:89 > > #9 0x000000000041627f in M3Syntax__TextSetPutList (M3_CT76zs_s=, M3_BxxOH1_wr=, > > M3_AicXUJ_quoted=) at ../src/M3Syntax.m3:129 > > #10 0x00000000004169cf in M3Syntax__EnumToText (M3_Bqnwti_e=, M3_ACGgGY_env=) at ../src/M3Syntax.m3:173 > > #11 0x000000000041791a in M3Syntax__UnstrToText (M3_AIPC2K_u=, M3_ACGgGY_env=) at ../src/M3Syntax.m3:295 > > #12 0x0000000000414ad4 in M3File__DumpInterface (M3_CjySNq_t=, M3_Bd56fi_dirPath=, > > M3_AicXUJ_updateMakefile=) at ../src/M3File.m3:78 > > #13 0x000000000040f3f9 in Fields__Do__MakeFieldsFiles.1150 () at ../AMD64_LINUX/Fields.m3:526 > > #14 0x000000000040a713 in Fields__Do (M3_DsEykq_xml=, M3_Bd56fi_versionString=, > > M3_Bd56fi_beginStringV=, M3_D9m5ya_fieldTbl=, M3_Bd56fi_Dir=, > > M3_Bd56fi_SrcDir=, M3_AZx9O5_binaryFields=, M3_CebKnt_specialFields=, > > M3_CT76zs_checkedTypes=) at ../AMD64_LINUX/Fields.m3:538 > > #15 0x0000000000413948 in Main_M3 (M3_AcxOUs_mode=) at ../AMD64_LINUX/Main.m3:431 > > #16 0x00000000004742ad in RTLinker__RunMainBody (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:408 > > #17 0x0000000000473638 in RTLinker__AddUnitI (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:115 > > #18 0x00000000004736cc in RTLinker__AddUnit (M3_DjPxE5_b=) at ../src/runtime/common/RTLinker.m3:124 > > #19 0x0000000000405df8 in main (argc=8, argv=0x7fffffffe2b8, envp=0x7fffffffe300) at _m3main.c:22 > > (gdb) > > > > I believe all the TextStats operations are killing the performance. Can we remove them? > > > > Here is SetChars: > > > > PROCEDURE SetChars (VAR a: ARRAY OF CHAR; t: T; start: CARDINAL) = > > BEGIN > > TextStats.NoteGround (TextStats.Op.SetChars); > > TextStats.NoteGround (TextStats.Op.get_chars); > > t.get_chars (a, start); > > TextStats.NoteFinished (TextStats.Op.get_chars); > > TextStats.NoteFinished (TextStats.Op.SetChars); > > END SetChars; > > > > I'm not sure how best to do this. Removing them without killing functionality might not be > > easy. Looks like a case for conditional compilation. > > Now the instrumentation calls are commented out with (*47 ... 74*), which could be reinstated > by string search & replace "(*47" -> "(*47*)", etc., and removed again similarly. These are > only needed for performance study of the algorithms, which likely won't be done often. > > > > > Maybe generate the .m3s from some other source, with and without the TextStats, depending on > > a setting in m3makefile? > > > > Second best would be to have a boolean flag in Text.m3 so the TextStats never get called. > > > > Mika > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Fri Jan 10 16:27:10 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Fri, 10 Jan 2014 07:27:10 -0800 Subject: [M3devel] performance problems in CM3 Text operations on AMD64_LINUX In-Reply-To: References: <20140104185511.9000A1A2080@async.async.caltech.edu>, <52CF0E43.2030701@lcwb.coop> Message-ID: <20140110152711.036F21A208F@async.async.caltech.edu> >Mika=2C how did you find this?Control-c every so often and look at callstac= >k?Or a lot of stepping? "Poor man's profiling" -- since profiling is usually broken in Modula-3. It worked back with SRC M3 but that was a while ago :-) 1. run program in gdb 2. close eyes 3. count to three 4. hit ctrl-C 5. open eyes (repeat a few times) "close eyes" is important! From dragisha at m3w.org Sun Jan 12 09:49:23 2014 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 12 Jan 2014 09:49:23 +0100 Subject: [M3devel] Conversion to Git Message-ID: <4B139BF3-A08A-4FAB-A37B-C8FFCCDC1D7B@m3w.org> Hi, I am at this again? Last night I did another full conversion, and now I am verifying results. After verification I will do one more, final, conversion. For best results, I would like to have emails/names of following people/authors. Please reply directly to me with this data. alexb darko dbenavid dragisha eiserlohpp hosking hudson jkrell khaeusler kschleiser m3 mand micha mika mm neels pmckinna rcoleburn rforb rillig rodney sk stsp thielema uamoore wagner -- Dragi?a Duri? dragisha at m3w.org -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Sun Jan 12 18:53:17 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sun, 12 Jan 2014 09:53:17 -0800 Subject: [M3devel] CM3 crashing Message-ID: <20140112175317.9F0031A2080@async.async.caltech.edu> Hi m3devel, Anyone have an idea what's going on here? I have program that compiles with PM3 (FreeBSD4): ... new source -> compiling ../src/Main.m3 "../src/Main.m3", line 15: warning: exception is never raised: OSError.E "../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted "../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure new opaque info -> recompiling HParseStd.m3 ... With CM3 at head (AMD64_LINUX (userthreads)), the following happens: --- building in AMD64_LINUX --- echo _PARSER_SOURCES +='"'`pwd`/../src/H.t'"' > H_t.tmpl echo _PARSER_SOURCES +='"'`pwd`/../src/H.l'"' > H_l.tmpl echo _PARSER_SOURCES +='"'`pwd`/../src/H.y'"' > H_y.tmpl new source -> compiling Main.m3 "../src/Main.m3", line 104: warning: potentially unhandled exception: RTAllocator.OutOfMemory "../src/Main.m3", line 138: warning: potentially unhandled exception: RTAllocator.OutOfMemory "../src/Main.m3", line 15: warning: exception is never raised: OSError.E "../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted "../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure *** *** runtime error: *** Attempt to reference an illegal memory location. *** file "../src/thread/POSIX/ThreadPosix.m3", line 1073 *** Abort (gdb) run -x Starting program: /big/home/mika/cm3-boot2.userthreads/bin/cm3 -x [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". --- building in AMD64_LINUX --- echo _PARSER_SOURCES +='"'`pwd`/../src/H.t'"' > H_t.tmpl echo _PARSER_SOURCES +='"'`pwd`/../src/H.l'"' > H_l.tmpl echo _PARSER_SOURCES +='"'`pwd`/../src/H.y'"' > H_y.tmpl new source -> compiling Main.m3 "../src/Main.m3", line 104: warning: potentially unhandled exception: RTAllocator.OutOfMemory "../src/Main.m3", line 138: warning: potentially unhandled exception: RTAllocator.OutOfMemory "../src/Main.m3", line 15: warning: exception is never raised: OSError.E "../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted "../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure Program received signal SIGSEGV, Segmentation fault. 0x000000000072da7c in RTHooks__Concat (M3_Bd56fi_t=, M3_Bd56fi_u=) at ../src/text/TextCat.m3:36 36 u.get_info (RInfo); (gdb) where #0 0x000000000072da7c in RTHooks__Concat (M3_Bd56fi_t=, M3_Bd56fi_u=) at ../src/text/TextCat.m3:36 #1 0x00000000006904d1 in M3CG_Check__PutErr (M3_CquYO2_u=, M3_Bd56fi_a=, M3_Bd56fi_b=, M3_Bd56fi_c=) at ../src/M3CG_Check.m3:190 #2 0x0000000000692133 in M3CG_Check__CheckProc (M3_CquYO2_self=, M3_BgUkgw_p=) at ../src/M3CG_Check.m3:513 #3 0x00000000006969e7 in M3CG_Check__load_procedure (M3_CquYO2_self=, M3_BgUkgw_p=) at ../src/M3CG_Check.m3:1289 #4 0x000000000051c846 in CG__Load_procedure (M3_BgUkgw_p=) at ../src/misc/CG.m3:2547 #5 0x0000000000564b6d in Procedure__Load (M3_D4a7Im_t=) at ../src/values/Procedure.m3:358 #6 0x000000000054f81f in Value__Load (M3_EjfEr4_t=) at ../src/values/Value.m3:61 #7 0x00000000005b3451 in ProcExpr__Compile (M3_EeJ4NO_p=) at ../src/exprs/ProcExpr.m3:96 #8 0x0000000000596873 in Expr__Compile (M3_ES44mX_t=) at ../src/exprs/Expr.m3:145 #9 0x00000000005588db in Formal__GenProcedure (M3_DfGE7n_t=, M3_ES44mX_actual=, M3_ES44mX_proc=) at ../src/values/Formal.m3:550 #10 0x000000000055809c in Formal__EmitArg (M3_ES44mX_proc=, M3_EjfEr4_formal=, M3_ES44mX_actual=) at ../src/values/Formal.m3:459 #11 0x000000000054e82c in UserProc (M3_ChqBRb_ce=) at ../src/types/UserProc.m3:131 #12 0x00000000005a0de2 in CallExpr (M3_ChqBRb_p=) at ../src/exprs/CallExpr.m3:314 #13 0x0000000000596825 in Expr (M3_ES44mX_t=) at ../src/exprs/Expr.m3:138 #14 0x0000000000557e4d in Formal__PrepArg (M3_EjfEr4_formal=, M3_ES44mX_actual=) at ../src/values/Formal.m3:431 #15 0x000000000054e48e in UserProc (M3_ChqBRb_ce=) at ../src/types/UserProc.m3:91 #16 0x00000000005a0de2 in CallExpr (M3_ChqBRb_p=) at ../src/exprs/CallExpr.m3:314 #17 0x0000000000596825 in Expr (M3_ES44mX_t=) at ../src/exprs/Expr.m3:138 #18 0x00000000004f741e in Marker__EmitReturn (M3_ES44mX_expr=, M3_AicXUJ_fromFinally=) at ../src/misc/Marker.m3:540 #19 0x00000000005d9ed1 in ReturnStmt__Compile (M3_CnuIWR_p=) at ../src/stmts/ReturnStmt.m3:55 #20 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) at ../src/stmts/Stmt.m3:115 #21 0x00000000005dfdd5 in TypeCaseStmt__CompileCaseBody (M3_BVmtE4_p=, M3_CUZbO1_c=, M3_CFkyBH_ref=, M3_AcxOUs_case_label=, M3_AcxOUs_exit_label=, M3_EIM114_done=) at ../src/stmts/TypeCaseStmt.m3:345 #22 0x00000000005df778 in TypeCaseStmt__Compile (M3_BVmtE4_p=) at ../src/stmts/TypeCaseStmt.m3:284 #23 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) at ../src/stmts/Stmt.m3:115 #24 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=) at ../src/stmts/WithStmt.m3:150 #25 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) at ../src/stmts/Stmt.m3:115 #26 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=) at ../src/stmts/WithStmt.m3:150 #27 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) at ../src/stmts/Stmt.m3:115 #28 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=) at ../src/stmts/WithStmt.m3:150 #29 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) at ../src/stmts/Stmt.m3:115 #30 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=) at ../src/stmts/WithStmt.m3:150 #31 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) at ../src/stmts/Stmt.m3:115 #32 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=) at ../src/stmts/WithStmt.m3:150 #33 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) at ../src/stmts/Stmt.m3:115 #34 0x00000000005d7dd9 in IfStmt__Compile (M3_EFMY1O_p=) at ../src/stmts/IfStmt.m3:105 #35 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) at ../src/stmts/Stmt.m3:115 #36 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=) at ../src/stmts/WithStmt.m3:150 #37 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) at ../src/stmts/Stmt.m3:115 #38 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=) at ../src/stmts/WithStmt.m3:150 #39 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) at ../src/stmts/Stmt.m3:115 #40 0x00000000005dfcfa in TypeCaseStmt__CompileCaseBody (M3_BVmtE4_p=, M3_CUZbO1_c=, M3_CFkyBH_ref=, M3_AcxOUs_case_label=, M3_AcxOUs_exit_label=, M3_EIM114_done=) at ../src/stmts/TypeCaseStmt.m3:339 #41 0x00000000005df778 in TypeCaseStmt__Compile (M3_BVmtE4_p=) at ../src/stmts/TypeCaseStmt.m3:284 #42 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) at ../src/stmts/Stmt.m3:115 #43 0x00000000005ce5f3 in BlockStmt__Compile (M3_CwhzFF_p=) at ../src/stmts/BlockStmt.m3:103 #44 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) at ../src/stmts/Stmt.m3:115 #45 0x00000000005667a9 in Procedure__GenBody (M3_D4a7Im_p=) at ../src/values/Procedure.m3:577 #46 0x000000000056631d in Procedure__EmitBody (M3_CknzAQ_x=) at ../src/values/Procedure.m3:541 #47 0x0000000000509703 in ProcBody__EmitBody (M3_BswZye_t=) at ../src/misc/ProcBody.m3:170 #48 0x000000000050887f in ProcBody__EmitAll (M3_EN2A1V_proc_info=) at ../src/misc/ProcBody.m3:69 #49 0x00000000005603d4 in Module__CompileModule (M3_DZ1mTg_t=) at ../src/values/Module.m3:905 #50 0x000000000055fc79 in Module__Compile (M3_DZ1mTg_t=) at ../src/values/Module.m3:837 #51 0x00000000004ec678 in M3Front__DoCompile () at ../src/misc/M3Front.m3:218 #52 0x00000000004eb7e9 in M3Front__Compile (M3_Algh87_input=, M3_CT7GjM_env=, M3_BySCJy_options=) at ../src/misc/M3Front.m3:66 #53 0x0000000000411db9 in Builder__RunM3 (M3_Cwb8uG_s=, M3_AXLf8B_u=, M3_Bd56fi_object=) at ../src/Builder.m3:1705 #54 0x000000000040ff74 in Builder__PushOneM3 (M3_Cwb8uG_s=, M3_AXLf8B_u=) at ../src/Builder.m3:1359 #55 0x000000000040f5ca in Builder__CompileM3 (M3_Cwb8uG_s=, M3_AXLf8B_u=) at ../src/Builder.m3:1247 #56 0x000000000040de26 in Builder__CompileOne (M3_Cwb8uG_s=, M3_AXLf8B_u=) at ../src/Builder.m3:1077 #57 0x000000000040d7b7 in Builder__CompileEverything (M3_Cwb8uG_s=, M3_Cw4bpV_schedule=) at ../src/Builder.m3:1025 #58 0x0000000000408676 in Builder__CompileUnits (M3_Bd56fi_main=, M3_EEuw3X_units=, M3_C1FTrk_sys_libs=, M3_A2QN6Z_info_kind=, M3_An02H2_mach=) at ../src/Builder.m3:336 #59 0x0000000000406c0b in Builder__BuildPgm (M3_Bd56fi_prog=, M3_EEuw3X_units=, M3_C1FTrk_sys_libs=, M3_AicXUJ_shared=, M3_An02H2_m=) at ../src/Builder.m3:28 #60 0x00000000004256f3 in M3Build__BuildProgram (M3_ABp1Zk_t=, M3_DLS2Hj_nm=) at ../src/M3Build.m3:1515 #61 0x000000000042553b in M3Build__DoProgram (M3_An02H2_m=, M3_AcxOUs_n_args=) at ../src/M3Build.m3:1491 #62 0x00000000004d01ad in QMachine__DoCall (M3_An02H2_t=, M3_AcxOUs_n_args=, M3_AicXUJ_isFunc=, M3_AicXUJ_outer=) at ../src/QMachine.m3:546 #63 0x00000000004cee3f in QMachine__Eval (M3_An02H2_t=) at ../src/QMachine.m3:422 #64 0x00000000004d9152 in QMachine (M3_An02H2_t=, M3_Bd56fi_path=, M3_AicXUJ_from_code=) at ../src/QMachine.m3:1773 #65 0x00000000004d8f2f in QMachine (M3_An02H2_t=, M3_Bd56fi_path=) at ../src/QMachine.m3:1744 #66 0x0000000000420f0d in M3Build (M3_ABp1Zk_t=, M3_Bd56fi_file=) at ../src/M3Build.m3:679 #67 0x0000000000422dc9 in M3Build__DoIncludeDir (M3_An02H2_m=, M3_AcxOUs_n_args=) at ../src/M3Build.m3:1054 #68 0x00000000004d01ad in QMachine__DoCall (M3_An02H2_t=, M3_AcxOUs_n_args=, M3_AicXUJ_isFunc=, M3_AicXUJ_outer=) at ../src/QMachine.m3:546 #69 0x00000000004cee3f in QMachine__Eval (M3_An02H2_t=) at ../src/QMachine.m3:422 #70 0x00000000004cd167 in QMachine__Evaluate (M3_An02H2_t=, M3_CYwAos_s=) at ../src/QMachine.m3:165 #71 0x00000000004c3c18 in Quake__Run (M3_An02H2_m=, M3_Bd56fi_source_file=) at ../src/Quake.m3:19 #72 0x000000000041f10d in M3Build__Run (M3_ABp1Zk_t=, M3_Bd56fi_makefile=) at ../src/M3Build.m3:242 #73 0x00000000004372ed in Main__DoIt () at ../src/Main.m3:117 #74 0x0000000000437671 in Main_M3 (M3_AcxOUs_mode=) at ../src/Main.m3:231 #75 0x00000000007087ed in RTLinker__RunMainBody (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:408 #76 0x0000000000707b78 in RTLinker__AddUnitI (M3_DjPxE3_m=) at ../src/runtime/common/RTLinker.m3:115 #77 0x0000000000707c0c in RTLinker__AddUnit (M3_DjPxE5_b=) at ../src/runtime/common/RTLinker.m3:124 #78 0x0000000000405b48 in main (argc=2, argv=0x7fffffffe378, envp=0x7fffffffe390) at _m3main.c:22 (gdb) Looks like something very basic... From mika at async.caltech.edu Sun Jan 12 18:58:30 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sun, 12 Jan 2014 09:58:30 -0800 Subject: [M3devel] CM3 crashing In-Reply-To: <20140112175317.9F0031A2080@async.async.caltech.edu> References: <20140112175317.9F0031A2080@async.async.caltech.edu> Message-ID: <20140112175830.C51881A2080@async.async.caltech.edu> Sigh... 188 PROCEDURE PutErr (u: U; a, b, c: TEXT := NIL) = 189 BEGIN 190 u.note_error ("********* M3CG_Check ERROR *********** " & a & b & c); 191 u.child.comment ("********* M3CG_Check ERROR *********** ", a, b, c); 192 INC (u.n_errors); 193 END PutErr; Ah.... ok.... "../src/Main.m3", line 129: ********* M3CG_Check ERROR *********** NIL procedure**NIL****NIL** "../src/Main.m3", line 131: ********* M3CG_Check ERROR *********** NIL procedure**NIL****NIL** "../src/Main.m3", line 133: ********* M3CG_Check ERROR *********** NIL procedure**NIL****NIL** " The code is: HIntExpr.Xor => RETURN NewConst(CBitwise(av, bv, Word.Xor), ab) | HIntExpr.Bor => RETURN NewConst(CBitwise(av, bv, Word.Or), ab) | HIntExpr.Band => RETURN NewConst(CBitwise(av, bv, Word.And), ab) I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ... mika writes: >Hi m3devel, > >Anyone have an idea what's going on here? > >I have program that compiles with PM3 (FreeBSD4): > >... >new source -> compiling ../src/Main.m3 >"../src/Main.m3", line 15: warning: exception is never raised: OSError.E >"../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted >"../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure > >new opaque info -> recompiling HParseStd.m3 >... > >With CM3 at head (AMD64_LINUX (userthreads)), the following happens: > >--- building in AMD64_LINUX --- > >echo _PARSER_SOURCES +='"'`pwd`/../src/H.t'"' > H_t.tmpl >echo _PARSER_SOURCES +='"'`pwd`/../src/H.l'"' > H_l.tmpl >echo _PARSER_SOURCES +='"'`pwd`/../src/H.y'"' > H_y.tmpl >new source -> compiling Main.m3 >"../src/Main.m3", line 104: warning: potentially unhandled exception: RTAlloca >tor.OutOfMemory >"../src/Main.m3", line 138: warning: potentially unhandled exception: RTAlloca >tor.OutOfMemory >"../src/Main.m3", line 15: warning: exception is never raised: OSError.E >"../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted >"../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure > > >*** >*** runtime error: >*** Attempt to reference an illegal memory location. >*** file "../src/thread/POSIX/ThreadPosix.m3", line 1073 >*** > >Abort > >(gdb) run -x >Starting program: /big/home/mika/cm3-boot2.userthreads/bin/cm3 -x >[Thread debugging using libthread_db enabled] >Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". >--- building in AMD64_LINUX --- > >echo _PARSER_SOURCES +='"'`pwd`/../src/H.t'"' > H_t.tmpl >echo _PARSER_SOURCES +='"'`pwd`/../src/H.l'"' > H_l.tmpl >echo _PARSER_SOURCES +='"'`pwd`/../src/H.y'"' > H_y.tmpl >new source -> compiling Main.m3 >"../src/Main.m3", line 104: warning: potentially unhandled exception: RTAlloca >tor.OutOfMemory >"../src/Main.m3", line 138: warning: potentially unhandled exception: RTAlloca >tor.OutOfMemory >"../src/Main.m3", line 15: warning: exception is never raised: OSError.E >"../src/Main.m3", line 15: warning: exception is never raised: Thread.Alerted >"../src/Main.m3", line 15: warning: exception is never raised: Wr.Failure > >Program received signal SIGSEGV, Segmentation fault. >0x000000000072da7c in RTHooks__Concat (M3_Bd56fi_t=, M >3_Bd56fi_u=) at ../src/text/TextCat.m3:36 >36 u.get_info (RInfo); >(gdb) where >#0 0x000000000072da7c in RTHooks__Concat (M3_Bd56fi_t=>, M3_Bd56fi_u=) at ../src/text/TextCat.m3:36 >#1 0x00000000006904d1 in M3CG_Check__PutErr (M3_CquYO2_u=ble>, M3_Bd56fi_a=, M3_Bd56fi_b=e>, M3_Bd56fi_c=) at ../src/M3CG_Check.m3:190 >#2 0x0000000000692133 in M3CG_Check__CheckProc (M3_CquYO2_self= variable>, M3_BgUkgw_p=) at ../src/M3CG_Check.m3:513 >#3 0x00000000006969e7 in M3CG_Check__load_procedure (M3_CquYO2_self=ading variable>, M3_BgUkgw_p=) at ../src/M3CG_Check.m3 >:1289 >#4 0x000000000051c846 in CG__Load_procedure (M3_BgUkgw_p=ble>) at ../src/misc/CG.m3:2547 >#5 0x0000000000564b6d in Procedure__Load (M3_D4a7Im_t=>) at ../src/values/Procedure.m3:358 >#6 0x000000000054f81f in Value__Load (M3_EjfEr4_t=) a >t ../src/values/Value.m3:61 >#7 0x00000000005b3451 in ProcExpr__Compile (M3_EeJ4NO_p=le>) at ../src/exprs/ProcExpr.m3:96 >#8 0x0000000000596873 in Expr__Compile (M3_ES44mX_t=) > at ../src/exprs/Expr.m3:145 >#9 0x00000000005588db in Formal__GenProcedure (M3_DfGE7n_t=iable>, M3_ES44mX_actual=, M3_ES44mX_proc=ng variable>) at ../src/values/Formal.m3:550 >#10 0x000000000055809c in Formal__EmitArg (M3_ES44mX_proc=ble>, M3_EjfEr4_formal=, M3_ES44mX_actual=ng variable>) at ../src/values/Formal.m3:459 >#11 0x000000000054e82c in UserProc (M3_ChqBRb_ce=) at >../src/types/UserProc.m3:131 >#12 0x00000000005a0de2 in CallExpr (M3_ChqBRb_p=) at . >./src/exprs/CallExpr.m3:314 >#13 0x0000000000596825 in Expr (M3_ES44mX_t=) at ../sr >c/exprs/Expr.m3:138 >#14 0x0000000000557e4d in Formal__PrepArg (M3_EjfEr4_formal=iable>, M3_ES44mX_actual=) at ../src/values/Formal.m3: >431 >#15 0x000000000054e48e in UserProc (M3_ChqBRb_ce=) at >../src/types/UserProc.m3:91 >#16 0x00000000005a0de2 in CallExpr (M3_ChqBRb_p=) at . >./src/exprs/CallExpr.m3:314 >#17 0x0000000000596825 in Expr (M3_ES44mX_t=) at ../sr >c/exprs/Expr.m3:138 >#18 0x00000000004f741e in Marker__EmitReturn (M3_ES44mX_expr=riable>, M3_AicXUJ_fromFinally=) at ../src/misc/Marker >.m3:540 >#19 0x00000000005d9ed1 in ReturnStmt__Compile (M3_CnuIWR_p=able>) at ../src/stmts/ReturnStmt.m3:55 >#20 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) > at ../src/stmts/Stmt.m3:115 >#21 0x00000000005dfdd5 in TypeCaseStmt__CompileCaseBody (M3_BVmtE4_p=ading variable>, M3_CUZbO1_c=, M3_CFkyBH_ref=ading variable>, M3_AcxOUs_case_label=, > M3_AcxOUs_exit_label=, M3_EIM114_done=ng variable>) at ../src/stmts/TypeCaseStmt.m3:345 >#22 0x00000000005df778 in TypeCaseStmt__Compile (M3_BVmtE4_p=riable>) at ../src/stmts/TypeCaseStmt.m3:284 >#23 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) > at ../src/stmts/Stmt.m3:115 >#24 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=le>) at ../src/stmts/WithStmt.m3:150 >#25 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) > at ../src/stmts/Stmt.m3:115 >#26 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=le>) at ../src/stmts/WithStmt.m3:150 >#27 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) > at ../src/stmts/Stmt.m3:115 >#28 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=le>) at ../src/stmts/WithStmt.m3:150 >#29 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) > at ../src/stmts/Stmt.m3:115 >#30 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=le>) at ../src/stmts/WithStmt.m3:150 >#31 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) > at ../src/stmts/Stmt.m3:115 >#32 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=le>) at ../src/stmts/WithStmt.m3:150 >#33 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) > at ../src/stmts/Stmt.m3:115 >#34 0x00000000005d7dd9 in IfStmt__Compile (M3_EFMY1O_p=>) at ../src/stmts/IfStmt.m3:105 >#35 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) > at ../src/stmts/Stmt.m3:115 >#36 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=le>) at ../src/stmts/WithStmt.m3:150 >#37 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) > at ../src/stmts/Stmt.m3:115 >#38 0x00000000005e1822 in WithStmt__Compile (M3_BurbBt_p=le>) at ../src/stmts/WithStmt.m3:150 >#39 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) > at ../src/stmts/Stmt.m3:115 >#40 0x00000000005dfcfa in TypeCaseStmt__CompileCaseBody (M3_BVmtE4_p=ading variable>, M3_CUZbO1_c=, M3_CFkyBH_ref=ading variable>, M3_AcxOUs_case_label=, > M3_AcxOUs_exit_label=, M3_EIM114_done=ng variable>) at ../src/stmts/TypeCaseStmt.m3:339 >#41 0x00000000005df778 in TypeCaseStmt__Compile (M3_BVmtE4_p=riable>) at ../src/stmts/TypeCaseStmt.m3:284 >#42 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) > at ../src/stmts/Stmt.m3:115 >#43 0x00000000005ce5f3 in BlockStmt__Compile (M3_CwhzFF_p=ble>) at ../src/stmts/BlockStmt.m3:103 >#44 0x00000000005ccb57 in Stmt__Compile (M3_Bt9Ne6_t=) > at ../src/stmts/Stmt.m3:115 >#45 0x00000000005667a9 in Procedure__GenBody (M3_D4a7Im_p=ble>) at ../src/values/Procedure.m3:577 >#46 0x000000000056631d in Procedure__EmitBody (M3_CknzAQ_x=able>) at ../src/values/Procedure.m3:541 >#47 0x0000000000509703 in ProcBody__EmitBody (M3_BswZye_t=ble>) at ../src/misc/ProcBody.m3:170 >#48 0x000000000050887f in ProcBody__EmitAll (M3_EN2A1V_proc_info=g variable>) at ../src/misc/ProcBody.m3:69 >#49 0x00000000005603d4 in Module__CompileModule (M3_DZ1mTg_t=riable>) at ../src/values/Module.m3:905 >#50 0x000000000055fc79 in Module__Compile (M3_DZ1mTg_t=>) at ../src/values/Module.m3:837 >#51 0x00000000004ec678 in M3Front__DoCompile () at ../src/misc/M3Front.m3:218 >#52 0x00000000004eb7e9 in M3Front__Compile (M3_Algh87_input=iable>, M3_CT7GjM_env=, M3_BySCJy_options=ng variable>) at ../src/misc/M3Front.m3:66 >#53 0x0000000000411db9 in Builder__RunM3 (M3_Cwb8uG_s= >, M3_AXLf8B_u=, M3_Bd56fi_object=le>) at ../src/Builder.m3:1705 >#54 0x000000000040ff74 in Builder__PushOneM3 (M3_Cwb8uG_s=ble>, M3_AXLf8B_u=) at ../src/Builder.m3:1359 >#55 0x000000000040f5ca in Builder__CompileM3 (M3_Cwb8uG_s=ble>, M3_AXLf8B_u=) at ../src/Builder.m3:1247 >#56 0x000000000040de26 in Builder__CompileOne (M3_Cwb8uG_s=able>, M3_AXLf8B_u=) at ../src/Builder.m3:1077 >#57 0x000000000040d7b7 in Builder__CompileEverything (M3_Cwb8uG_s=ng variable>, M3_Cw4bpV_schedule=) at ../src/Builder.m >3:1025 >#58 0x0000000000408676 in Builder__CompileUnits (M3_Bd56fi_main= variable>, M3_EEuw3X_units=, M3_C1FTrk_sys_libs=r reading variable>, M3_A2QN6Z_info_kind=, > M3_An02H2_mach=) at ../src/Builder.m3:336 >#59 0x0000000000406c0b in Builder__BuildPgm (M3_Bd56fi_prog=iable>, M3_EEuw3X_units=, M3_C1FTrk_sys_libs=ading variable>, M3_AicXUJ_shared=, > M3_An02H2_m=) at ../src/Builder.m3:28 >#60 0x00000000004256f3 in M3Build__BuildProgram (M3_ABp1Zk_t=riable>, M3_DLS2Hj_nm=) at ../src/M3Build.m3:1515 >#61 0x000000000042553b in M3Build__DoProgram (M3_An02H2_m=ble>, M3_AcxOUs_n_args=) at ../src/M3Build.m3:1491 >#62 0x00000000004d01ad in QMachine__DoCall (M3_An02H2_t=e>, M3_AcxOUs_n_args=, M3_AicXUJ_isFunc= variable>, M3_AicXUJ_outer=) > at ../src/QMachine.m3:546 >#63 0x00000000004cee3f in QMachine__Eval (M3_An02H2_t= >) at ../src/QMachine.m3:422 >#64 0x00000000004d9152 in QMachine (M3_An02H2_t=, M3_B >d56fi_path=, M3_AicXUJ_from_code=le>) at ../src/QMachine.m3:1773 >#65 0x00000000004d8f2f in QMachine (M3_An02H2_t=, M3_B >d56fi_path=) at ../src/QMachine.m3:1744 >#66 0x0000000000420f0d in M3Build (M3_ABp1Zk_t=, M3_Bd >56fi_file=) at ../src/M3Build.m3:679 >#67 0x0000000000422dc9 in M3Build__DoIncludeDir (M3_An02H2_m=riable>, M3_AcxOUs_n_args=) at ../src/M3Build.m3:1054 >#68 0x00000000004d01ad in QMachine__DoCall (M3_An02H2_t=e>, M3_AcxOUs_n_args=, M3_AicXUJ_isFunc= variable>, M3_AicXUJ_outer=) > at ../src/QMachine.m3:546 >#69 0x00000000004cee3f in QMachine__Eval (M3_An02H2_t= >) at ../src/QMachine.m3:422 >#70 0x00000000004cd167 in QMachine__Evaluate (M3_An02H2_t=ble>, M3_CYwAos_s=) at ../src/QMachine.m3:165 >#71 0x00000000004c3c18 in Quake__Run (M3_An02H2_m=, M3 >_Bd56fi_source_file=) at ../src/Quake.m3:19 >#72 0x000000000041f10d in M3Build__Run (M3_ABp1Zk_t=, >M3_Bd56fi_makefile=) at ../src/M3Build.m3:242 >#73 0x00000000004372ed in Main__DoIt () at ../src/Main.m3:117 >#74 0x0000000000437671 in Main_M3 (M3_AcxOUs_mode=) at > ../src/Main.m3:231 >#75 0x00000000007087ed in RTLinker__RunMainBody (M3_DjPxE3_m=riable>) at ../src/runtime/common/RTLinker.m3:408 >#76 0x0000000000707b78 in RTLinker__AddUnitI (M3_DjPxE3_m=ble>) at ../src/runtime/common/RTLinker.m3:115 >#77 0x0000000000707c0c in RTLinker__AddUnit (M3_DjPxE5_b=le>) at ../src/runtime/common/RTLinker.m3:124 >#78 0x0000000000405b48 in main (argc=2, argv=0x7fffffffe378, envp=0x7fffffffe3 >90) at _m3main.c:22 >(gdb) > >Looks like something very basic... > > From hosking at cs.purdue.edu Sun Jan 12 21:30:34 2014 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 12 Jan 2014 15:30:34 -0500 Subject: [M3devel] CM3 crashing In-Reply-To: <20140112175830.C51881A2080@async.async.caltech.edu> References: <20140112175317.9F0031A2080@async.async.caltech.edu> <20140112175830.C51881A2080@async.async.caltech.edu> Message-ID: Are you saying passing these used to work in PM3? Sounds like a front-end bug. I?m curious what changed to break it. On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: > > The code is: > > HIntExpr.Xor => RETURN NewConst(CBitwise(av, bv, Word.Xor), ab) > | > HIntExpr.Bor => RETURN NewConst(CBitwise(av, bv, Word.Or), ab) > | > HIntExpr.Band => RETURN NewConst(CBitwise(av, bv, Word.And), ab) > > I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ... Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Mobile +1 765 427 5484 From dragisha at m3w.org Sun Jan 12 22:45:32 2014 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 12 Jan 2014 22:45:32 +0100 Subject: [M3devel] Conversion to Git In-Reply-To: <4B139BF3-A08A-4FAB-A37B-C8FFCCDC1D7B@m3w.org> References: <4B139BF3-A08A-4FAB-A37B-C8FFCCDC1D7B@m3w.org> Message-ID: Thanks to everybody who send me infomation, esp. to Peter Eiserloh who gave me information + idea how to fill rest. Missing are emails from some Elego people and I expect I will have that soon. I will keep you all posted on conversion status. Current version is on Github (dragisha/cm3), and I am checking how it matches with local CVS checkouts. On 12 Jan 2014, at 09:49, Dragi?a Duri? wrote: > Hi, > > I am at this again? Last night I did another full conversion, and now I am verifying results. After verification I will do one more, final, conversion. > > For best results, I would like to have emails/names of following people/authors. Please reply directly to me with this data. > > alexb > darko > dbenavid > dragisha > eiserlohpp > hosking > hudson > jkrell > khaeusler > kschleiser > m3 > mand > micha > mika > mm > neels > pmckinna > rcoleburn > rforb > rillig > rodney > sk > stsp > thielema > uamoore > wagner > -- > Dragi?a Duri? > dragisha at m3w.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Mon Jan 13 01:25:39 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sun, 12 Jan 2014 16:25:39 -0800 Subject: [M3devel] CM3 crashing In-Reply-To: References: <20140112175317.9F0031A2080@async.async.caltech.edu> <20140112175830.C51881A2080@async.async.caltech.edu> Message-ID: <20140113002539.EC4EE1A2080@async.async.caltech.edu> Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3). I figured there aren't actually implementations of Word.Xor, Word.Or, and Word.And as procedures but they get inlined somehow? BTW, any idea about what's wrong with pthreads? Do you think the issue is with the garbage collector or with the threads, off the top of your head? Tony Hosking writes: >Are you saying passing these used to work in PM3? >Sounds like a front-end bug. I=92m curious what changed to break it. > >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: > >>=20 >> The code is: >>=20 >> HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Xor), ab) >> | >> HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Or), ab) >> | >> HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, = >Word.And), ab) >>=20 >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20 > > > >Antony Hosking | Associate Professor | Computer Science | Purdue = >University >305 N. University Street | West Lafayette | IN 47907 | USA >Mobile +1 765 427 5484 > > > > From hosking at cs.purdue.edu Mon Jan 13 16:46:36 2014 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 13 Jan 2014 10:46:36 -0500 Subject: [M3devel] CM3 crashing In-Reply-To: <20140113002539.EC4EE1A2080@async.async.caltech.edu> References: <20140112175317.9F0031A2080@async.async.caltech.edu> <20140112175830.C51881A2080@async.async.caltech.edu> <20140113002539.EC4EE1A2080@async.async.caltech.edu> Message-ID: <09749A02-B9B9-4C74-8CB4-73379E91385B@cs.purdue.edu> On Jan 12, 2014, at 7:25 PM, mika at async.caltech.edu wrote: > Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3). Interesting. Needs exploration. This should be filed as a bug. The workaround is to wrap the builtin calls (Word.Xor, etc.) in a local procedure so that you have an addressable procedure. > I figured there aren't actually implementations of Word.Xor, Word.Or, > and Word.And as procedures but they get inlined somehow? Yes, they are inlined, but there *is* an implementation in Word.m3 (generated from GenWord.mg). > BTW, any idea about what's wrong with pthreads? Do you think the issue > is with the garbage collector or with the threads, off the top of your > head? It must be the threads implementation, since the same collector is used with both pthreads and user threads, and I believe that user threads don?t suffer from this problem. My intuition is that it may be a problem with getting stack roots on particular platforms, probably because the stack bounds are not being computed correctly for a given architecture, or perhaps because reference alignment is not being observed correctly. For example, is it possible on x86_64 to have references stored in the stack aligned on 32-bit instead of 64-bit boundaries? I notice that RTThread.i3 says the pointer alignment is BYTESIZE(ADDRESS) on all platforms. Once upon a time the alignment was platform-dependent, so I wonder if we lost something somewhere with all the backend and porting work. It may be that the backend needs to be instructed to align all references appropriately in the stacks. One way to test all this might be to see if the thread stress tester causes things to break on older 32-bit platforms, where I expect all pointers will be aligned properly. Also, I?ve not inspected the code in ThreadPThreadC.c recently (it?s changed since I last looked), but I wonder if the stack bounds are being computed correctly so that the collector can find all the stack roots. I?d really like to get to the bottom of this problem. > Tony Hosking writes: >> Are you saying passing these used to work in PM3? >> Sounds like a front-end bug. I=92m curious what changed to break it. >> >> On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: >> >>> =20 >>> The code is: >>> =20 >>> HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, = >> Word.Xor), ab) >>> | >>> HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, = >> Word.Or), ab) >>> | >>> HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, = >> Word.And), ab) >>> =20 >>> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20 >> >> >> >> Antony Hosking | Associate Professor | Computer Science | Purdue = >> University >> 305 N. University Street | West Lafayette | IN 47907 | USA >> Mobile +1 765 427 5484 >> >> >> >> From hosking at cs.purdue.edu Mon Jan 13 17:03:38 2014 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 13 Jan 2014 11:03:38 -0500 Subject: [M3devel] CM3 crashing In-Reply-To: References: <20140112175317.9F0031A2080@async.async.caltech.edu> <20140112175830.C51881A2080@async.async.caltech.edu> <20140113002539.EC4EE1A2080@async.async.caltech.edu> Message-ID: Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?). In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem. As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC. Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning. Can someone confirm? If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed. On Jan 12, 2014, at 10:54 PM, Peter McKinna wrote: > Hey, > > I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. > My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped the second call to p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one. > It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. > > Regards Peter > > > > On Mon, Jan 13, 2014 at 11:25 AM, wrote: > Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3). > > I figured there aren't actually implementations of Word.Xor, Word.Or, > and Word.And as procedures but they get inlined somehow? > > BTW, any idea about what's wrong with pthreads? Do you think the issue > is with the garbage collector or with the threads, off the top of your > head? > > Tony Hosking writes: > >Are you saying passing these used to work in PM3? > >Sounds like a front-end bug. I=92m curious what changed to break it. > > > >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: > > > >>=20 > >> The code is: > >>=20 > >> HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, = > >Word.Xor), ab) > >> | > >> HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, = > >Word.Or), ab) > >> | > >> HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, = > >Word.And), ab) > >>=20 > >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20 > > > > > > > >Antony Hosking | Associate Professor | Computer Science | Purdue = > >University > >305 N. University Street | West Lafayette | IN 47907 | USA > >Mobile +1 765 427 5484 > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcolebur at SCIRES.COM Mon Jan 13 20:16:41 2014 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Mon, 13 Jan 2014 19:16:41 +0000 Subject: [M3devel] CM3 crashing Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D152@ATLEX04-SRV.SCIRES.LOCAL> Tony et al: The thread test program fails for me on 64-bit Windows 7. I've listed output from a couple of runs below: C:\cm3\Sandbox>cm3 --version Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2014-01-07 06:55:14 configuration: C:\cm3\bin\cm3.cfg host: NT386 target: NT386 FIRST RUN: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\runtime\common\RTCollector.m3", line 418 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** ...this last assertion repeats until you press CTRL-C to abort the program... SECOND RUN: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) .... *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x776f22d2 *** Stack trace: FP PC Procedure --------- --------- ------------------------------- 0xa6f9a8 0xc6afeb SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3 0xa6f9d0 0x776f22d2 0xa6f9e8 0xc4a39b LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3 0xa6fa10 0xc299f8 GetChar + 0x28 in ..\src\rw\Rd.m3 0xa6fb48 0xc213a8 RApply + 0x1b8 in ..\src\Main.m3 0xa6fb84 0xc4b71b ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3 0xa6fb90 0x76db336a 0xa6fbd0 0x7770bf32 ......... ......... ... more frames ... Maybe this info will be useful in tracking down the problem. --Randy Randy C. Coleburn, CISSP-ISSEP, GCED Corporate Information Security Officer (CISO) Scientific Research Corporation 2300 Windy Ridge Parkway, Suite 400 South, Atlanta, Georgia 30339 voice: (770) 989-9464, email: RColeburn at SciRes.com, fax: (770) 989-9497 [logo_SRC] "Technology Driven-Customer Focused" Quality Policy: "SRC is committed to delivering continually improving research & engineering excellence that meets or exceeds customer requirements." CONFIDENTIALITY NOTICE: This email constitutes an electronic communication within the meaning of the Electronic Communications Privacy Act, 18 U.S.C. 2510, and its disclosure is strictly limited to the named recipient(s) intended by the sender of this message. This email, and any attachments, may contain confidential and/or proprietary information of Scientific Research Corporation. If you are not a named recipient, any copying, using, disclosing or distributing to others the information in this email and attachments is STRICTLY PROHIBITED. If you have received this email in error, please notify the sender immediately and permanently delete the email, any attachments, and all copies thereof from any drives or storage media and destroy any printouts or hard copies of the email and attachments. EXPORT COMPLIANCE NOTICE: This email and any attachments may contain technical data subject to U.S export restrictions under the International Traffic in Arms Regulations (ITAR) or the Export Administration Regulations (EAR). Export or transfer of this technical data and/or related information to any foreign person(s) or entity(ies), either within the U.S. or outside of the U.S., may require advance export authorization by the appropriate U.S. Government agency prior to export or transfer. In addition, technical data may not be exported or transferred to certain countries or specified designated nationals identified by U.S. embargo controls without prior export authorization. By accepting this email and any attachments, all recipients confirm that they understand and will comply with all applicable ITAR, EAR and embargo compliance requirements. From: Tony Hosking [mailto:hosking at cs.purdue.edu] Sent: Monday, January 13, 2014 11:04 AM To: Peter McKinna Cc: m3devel Subject: EXT:Re: [M3devel] CM3 crashing Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?). In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem. As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC. Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning. Can someone confirm? If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed. On Jan 12, 2014, at 10:54 PM, Peter McKinna > wrote: Hey, I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped the second call to p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one. It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. Regards Peter On Mon, Jan 13, 2014 at 11:25 AM, > wrote: Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3). I figured there aren't actually implementations of Word.Xor, Word.Or, and Word.And as procedures but they get inlined somehow? BTW, any idea about what's wrong with pthreads? Do you think the issue is with the garbage collector or with the threads, off the top of your head? Tony Hosking writes: >Are you saying passing these used to work in PM3? >Sounds like a front-end bug. I=92m curious what changed to break it. > >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: > >>=20 >> The code is: >>=20 >> HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Xor), ab) >> | >> HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Or), ab) >> | >> HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, = >Word.And), ab) >>=20 >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20 > > > >Antony Hosking | Associate Professor | Computer Science | Purdue = >University >305 N. University Street | West Lafayette | IN 47907 | USA >Mobile +1 765 427 5484 > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.jpg Type: image/jpeg Size: 11339 bytes Desc: image002.jpg URL: From rcolebur at SCIRES.COM Mon Jan 13 20:55:44 2014 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Mon, 13 Jan 2014 19:55:44 +0000 Subject: [M3devel] CM3 crashing In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D152@ATLEX04-SRV.SCIRES.LOCAL> References: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D152@ATLEX04-SRV.SCIRES.LOCAL> Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D21A@ATLEX04-SRV.SCIRES.LOCAL> I've also tested on 32-bit Windows XP and the thread test program crashes there also. C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386 C:\cm3\Sandbox\cm3\m3-libs>cm3 --version Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2013-12-17 17:28:52 configuration: C:\cm3\bin\cm3.cfg host: NT386 target: NT386 Here is the output: C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Coleburn, Randy Sent: Monday, January 13, 2014 2:17 PM To: Tony Hosking; Peter McKinna Cc: m3devel Subject: Re: [M3devel] CM3 crashing Tony et al: The thread test program fails for me on 64-bit Windows 7. I've listed output from a couple of runs below: C:\cm3\Sandbox>cm3 --version Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2014-01-07 06:55:14 configuration: C:\cm3\bin\cm3.cfg host: NT386 target: NT386 FIRST RUN: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\runtime\common\RTCollector.m3", line 418 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** ...this last assertion repeats until you press CTRL-C to abort the program... SECOND RUN: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) .... *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x776f22d2 *** Stack trace: FP PC Procedure --------- --------- ------------------------------- 0xa6f9a8 0xc6afeb SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3 0xa6f9d0 0x776f22d2 0xa6f9e8 0xc4a39b LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3 0xa6fa10 0xc299f8 GetChar + 0x28 in ..\src\rw\Rd.m3 0xa6fb48 0xc213a8 RApply + 0x1b8 in ..\src\Main.m3 0xa6fb84 0xc4b71b ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3 0xa6fb90 0x76db336a 0xa6fbd0 0x7770bf32 ......... ......... ... more frames ... Maybe this info will be useful in tracking down the problem. --Randy From: Tony Hosking [mailto:hosking at cs.purdue.edu] Sent: Monday, January 13, 2014 11:04 AM To: Peter McKinna Cc: m3devel Subject: EXT:Re: [M3devel] CM3 crashing Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?). In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem. As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC. Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning. Can someone confirm? If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed. On Jan 12, 2014, at 10:54 PM, Peter McKinna > wrote: Hey, I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped the second call to p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one. It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. Regards Peter On Mon, Jan 13, 2014 at 11:25 AM, > wrote: Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3). I figured there aren't actually implementations of Word.Xor, Word.Or, and Word.And as procedures but they get inlined somehow? BTW, any idea about what's wrong with pthreads? Do you think the issue is with the garbage collector or with the threads, off the top of your head? Tony Hosking writes: >Are you saying passing these used to work in PM3? >Sounds like a front-end bug. I=92m curious what changed to break it. > >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: > >>=20 >> The code is: >>=20 >> HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Xor), ab) >> | >> HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Or), ab) >> | >> HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, = >Word.And), ab) >>=20 >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20 > > > >Antony Hosking | Associate Professor | Computer Science | Purdue = >University >305 N. University Street | West Lafayette | IN 47907 | USA >Mobile +1 765 427 5484 > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Mon Jan 13 21:02:57 2014 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 13 Jan 2014 15:02:57 -0500 Subject: [M3devel] CM3 crashing In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D21A@ATLEX04-SRV.SCIRES.LOCAL> References: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D152@ATLEX04-SRV.SCIRES.LOCAL> <0BB8FA59C2932741A3A2941A8B9D8BFF9251D21A@ATLEX04-SRV.SCIRES.LOCAL> Message-ID: <227E8C61-B070-49FF-8AD5-E23420E9C1B0@cs.purdue.edu> Is this from the head of source? Also, please run with @M3paranoidgc flag. On Jan 13, 2014, at 2:55 PM, Coleburn, Randy wrote: > I?ve also tested on 32-bit Windows XP and the thread test program crashes there also. > > C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386 > > C:\cm3\Sandbox\cm3\m3-libs>cm3 --version > Critical Mass Modula-3 version d5.9.0 > last updated: 2010-07-21 > compiled: 2013-12-17 17:28:52 > configuration: C:\cm3\bin\cm3.cfg > host: NT386 > target: NT386 > > Here is the output: > > C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help > Writing file...done > Creating read threads...done > Creating fork threads...done > Creating alloc threads...done > Creating lock threads...done > running...printing oldest/median age/newest > > *** > *** runtime error: > *** Attempt to reference an illegal memory location. > *** pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3 > *** > > *** > *** runtime error: > *** <*ASSERT*> failed. > *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 > *** > > *** > *** runtime error: > *** <*ASSERT*> failed. > *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 > *** > > This last assert repeats indefinitely until you press CTRL-C. > > --Randy > > > From: Coleburn, Randy > Sent: Monday, January 13, 2014 2:17 PM > To: Tony Hosking; Peter McKinna > Cc: m3devel > Subject: Re: [M3devel] CM3 crashing > > Tony et al: > > The thread test program fails for me on 64-bit Windows 7. > > I?ve listed output from a couple of runs below: > > C:\cm3\Sandbox>cm3 --version > Critical Mass Modula-3 version d5.9.0 > last updated: 2010-07-21 > compiled: 2014-01-07 06:55:14 > configuration: C:\cm3\bin\cm3.cfg > host: NT386 > target: NT386 > > FIRST RUN: > > C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe > Writing file...done > Creating read threads...done > Creating fork threads...done > Creating alloc threads...done > Creating lock threads...done > running...printing oldest/median age/newest > > *** > *** runtime error: > *** Attempt to reference an illegal memory location. > > *** > *** runtime error: > *** Attempt to reference an illegal memory location. > > *** > *** runtime error: > *** Attempt to reference an illegal memory location. > > *** > *** runtime error: > *** Attempt to reference an illegal memory location. > > *** > *** runtime error: > *** Attempt to reference an illegal memory location. > > *** > *** runtime error: > *** <*ASSERT*> failed. > *** file "..\src\runtime\common\RTCollector.m3", line 418 > *** > > *** > *** runtime error: > *** <*ASSERT*> failed. > *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 > *** > > *** > *** runtime error: > *** <*ASSERT*> failed. > *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 > *** > > ?this last assertion repeats until you press CTRL-C to abort the program? > > SECOND RUN: > > C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help > Writing file...done > Creating read threads...done > Creating fork threads...done > Creating alloc threads...done > Creating lock threads...done > running...printing oldest/median age/newest > ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) > .... > > *** > *** runtime error: > *** Attempt to reference an illegal memory location. > *** pc = 0x776f22d2 > *** > > Stack trace: > FP PC Procedure > --------- --------- ------------------------------- > 0xa6f9a8 0xc6afeb SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3 > 0xa6f9d0 0x776f22d2 > 0xa6f9e8 0xc4a39b LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3 > 0xa6fa10 0xc299f8 GetChar + 0x28 in ..\src\rw\Rd.m3 > 0xa6fb48 0xc213a8 RApply + 0x1b8 in ..\src\Main.m3 > 0xa6fb84 0xc4b71b ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3 > 0xa6fb90 0x76db336a > 0xa6fbd0 0x7770bf32 > ......... ......... ... more frames ... > > Maybe this info will be useful in tracking down the problem. > > --Randy > > > From: Tony Hosking [mailto:hosking at cs.purdue.edu] > Sent: Monday, January 13, 2014 11:04 AM > To: Peter McKinna > Cc: m3devel > Subject: EXT:Re: [M3devel] CM3 crashing > > Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?). > In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem. > > As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC. Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning. Can someone confirm? > > If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed. > > On Jan 12, 2014, at 10:54 PM, Peter McKinna wrote: > > > Hey, > > I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. > My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped the second call to p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one. > It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. > > Regards Peter > > > > On Mon, Jan 13, 2014 at 11:25 AM, wrote: > Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3). > > I figured there aren't actually implementations of Word.Xor, Word.Or, > and Word.And as procedures but they get inlined somehow? > > BTW, any idea about what's wrong with pthreads? Do you think the issue > is with the garbage collector or with the threads, off the top of your > head? > > Tony Hosking writes: > >Are you saying passing these used to work in PM3? > >Sounds like a front-end bug. I=92m curious what changed to break it. > > > >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: > > > >>=20 > >> The code is: > >>=20 > >> HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, = > >Word.Xor), ab) > >> | > >> HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, = > >Word.Or), ab) > >> | > >> HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, = > >Word.And), ab) > >>=20 > >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20 > > > > > > > >Antony Hosking | Associate Professor | Computer Science | Purdue = > >University > >305 N. University Street | West Lafayette | IN 47907 | USA > >Mobile +1 765 427 5484 > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcolebur at SCIRES.COM Mon Jan 13 21:13:55 2014 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Mon, 13 Jan 2014 20:13:55 +0000 Subject: [M3devel] CM3 crashing Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D28A@ATLEX04-SRV.SCIRES.LOCAL> Tony: Yes, it is from the HEAD as of 17 December 2013. Using the @M3paranoidgc flag, On 32-bit Windows XP I get: C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest . *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x431755 = RefSanityCheck + 0x2c in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. On 64-bit Windows 7 I get: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest . *** *** runtime error: *** An array subscript was out of range. *** file "..\src\rw\FileRd.m3", line 83 *** *** *** runtime error: *** An enumeration or subrange value was out of range. *** file "..\src\runtime\common\RTType.m3", line 71 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Tony Hosking [mailto:hosking at cs.purdue.edu] Sent: Monday, January 13, 2014 3:03 PM To: Coleburn, Randy Cc: Peter McKinna; m3devel Subject: EXT:Re: [M3devel] CM3 crashing Is this from the head of source? Also, please run with @M3paranoidgc flag. On Jan 13, 2014, at 2:55 PM, Coleburn, Randy > wrote: I've also tested on 32-bit Windows XP and the thread test program crashes there also. C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386 C:\cm3\Sandbox\cm3\m3-libs>cm3 --version Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2013-12-17 17:28:52 configuration: C:\cm3\bin\cm3.cfg host: NT386 target: NT386 Here is the output: C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Coleburn, Randy Sent: Monday, January 13, 2014 2:17 PM To: Tony Hosking; Peter McKinna Cc: m3devel Subject: Re: [M3devel] CM3 crashing Tony et al: The thread test program fails for me on 64-bit Windows 7. I've listed output from a couple of runs below: C:\cm3\Sandbox>cm3 --version Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2014-01-07 06:55:14 configuration: C:\cm3\bin\cm3.cfg host: NT386 target: NT386 FIRST RUN: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\runtime\common\RTCollector.m3", line 418 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** ...this last assertion repeats until you press CTRL-C to abort the program... SECOND RUN: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) .... *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x776f22d2 *** Stack trace: FP PC Procedure --------- --------- ------------------------------- 0xa6f9a8 0xc6afeb SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3 0xa6f9d0 0x776f22d2 0xa6f9e8 0xc4a39b LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3 0xa6fa10 0xc299f8 GetChar + 0x28 in ..\src\rw\Rd.m3 0xa6fb48 0xc213a8 RApply + 0x1b8 in ..\src\Main.m3 0xa6fb84 0xc4b71b ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3 0xa6fb90 0x76db336a 0xa6fbd0 0x7770bf32 ......... ......... ... more frames ... Maybe this info will be useful in tracking down the problem. --Randy From: Tony Hosking [mailto:hosking at cs.purdue.edu] Sent: Monday, January 13, 2014 11:04 AM To: Peter McKinna Cc: m3devel Subject: EXT:Re: [M3devel] CM3 crashing Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?). In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem. As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC. Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning. Can someone confirm? If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed. On Jan 12, 2014, at 10:54 PM, Peter McKinna > wrote: Hey, I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped the second call to p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one. It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. Regards Peter On Mon, Jan 13, 2014 at 11:25 AM, > wrote: Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3). I figured there aren't actually implementations of Word.Xor, Word.Or, and Word.And as procedures but they get inlined somehow? BTW, any idea about what's wrong with pthreads? Do you think the issue is with the garbage collector or with the threads, off the top of your head? Tony Hosking writes: >Are you saying passing these used to work in PM3? >Sounds like a front-end bug. I=92m curious what changed to break it. > >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: > >>=20 >> The code is: >>=20 >> HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Xor), ab) >> | >> HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Or), ab) >> | >> HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, = >Word.And), ab) >>=20 >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20 > > > >Antony Hosking | Associate Professor | Computer Science | Purdue = >University >305 N. University Street | West Lafayette | IN 47907 | USA >Mobile +1 765 427 5484 > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcolebur at SCIRES.COM Mon Jan 13 22:02:52 2014 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Mon, 13 Jan 2014 21:02:52 +0000 Subject: [M3devel] CM3 crashing In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D28A@ATLEX04-SRV.SCIRES.LOCAL> References: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D28A@ATLEX04-SRV.SCIRES.LOCAL> Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D3DF@ATLEX04-SRV.SCIRES.LOCAL> Tony: Here are a couple more runs on 64-bit Windows 7 that yield different results, albeit using the "-verbose" option. These may give some more clues. I note that in both cases it appears a read thread and a fork thread get the same ID (0) which looks very suspicious. By my read of the code (WITH id = i * nPer + j), I think the 2nd fork thread should be ID=10, not ID=0. Oops, now I see the problem: there is a bug in the PutInt procedure. The test (IF c > 10) should read instead (IF c >= 10). I'll commit a fix for that momentarily. But this fix just solves the output problem; it doesn't affect the fact that the program is misbehaving and crashing. RUN #1: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc -verbose Writing file...done Creating read threads... read=0 read=1 read=2 done Creating fork threads... fork=9 fork=0 fork=11 done Creating alloc threads... alloc=15 alloc=16 alloc=17 done Creating lock threads... lock=21 lock=22 lock=23 done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0xc286e0 = Init + 0x79 in ..\src\rw\FileRd.m3 *** Stack trace: FP PC Procedure --------- --------- ------------------------------- 0x7cfd60 0xc6afeb SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3 0x7cfda8 0xc286e0 Init + 0x79 in ..\src\rw\FileRd.m3 0x7cfdd4 0xc2865d Open + 0x4d in ..\src\rw\FileRd.m3 0x7cff0c 0xc21350 RApply + 0x160 in ..\src\Main.m3 0x7cff48 0xc4b71b ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3 0x7cff54 0x76db336a 0x7cff94 0x7770bf32 ......... ......... ... more frames ... RUN #2: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe -verbose Writing file...done Creating read threads... read=0 read=1 read=2 done Creating fork threads... fork=9 fork=0 fork=11 done Creating alloc threads... alloc=15 alloc=16 alloc=17 done Creating lock threads... lock=21 lock=22 lock=23 done running...printing oldest/median age/newest ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 534 loops. read Thread 1 completed 606 loops. read Thread 2 completed 398 loops. fork Thread 9 completed 8 loops. fork Thread 10 completed 8 loops. fork Thread 11 completed 8 loops. alloc Thread 15 completed 18296 loops. alloc Thread 16 completed 44871 loops. alloc Thread 17 completed 79125 loops. lock Thread 21 completed 3845080 loops. lock Thread 22 completed 2537613 loops. lock Thread 23 completed 4506151 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 790 loops. read Thread 1 completed 786 loops. read Thread 2 completed 675 loops. fork Thread 9 completed 9 loops. fork Thread 10 completed 9 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 22091 loops. alloc Thread 16 completed 47532 loops. alloc Thread 17 completed 68901 loops. lock Thread 21 completed 4560731 loops. lock Thread 22 completed 3440795 loops. lock Thread 23 completed 6433538 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 725 loops. read Thread 1 completed 705 loops. read Thread 2 completed 617 loops. fork Thread 9 completed 10 loops. fork Thread 10 completed 9 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 18560 loops. alloc Thread 16 completed 44487 loops. alloc Thread 17 completed 87219 loops. lock Thread 21 completed 3769840 loops. lock Thread 22 completed 3037581 loops. lock Thread 23 completed 6097922 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 825 loops. read Thread 1 completed 802 loops. read Thread 2 completed 669 loops. fork Thread 9 completed 9 loops. fork Thread 10 completed 10 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 16877 loops. alloc Thread 16 completed 53277 loops. alloc Thread 17 completed 77986 loops. lock Thread 21 completed 4087218 loops. lock Thread 22 completed 3116357 loops. lock Thread 23 completed 6466214 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 684 loops. read Thread 1 completed 686 loops. read Thread 2 completed 648 loops. fork Thread 9 completed 10 loops. fork Thread 10 completed 9 loops. fork Thread 11 completed 10 loops. alloc Thread 15 completed 19717 loops. alloc Thread 16 completed 47797 loops. alloc Thread 17 completed 78110 loops. lock Thread 21 completed 4580435 loops. lock Thread 22 completed 2974247 loops. lock Thread 23 completed 5843850 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 748 loops. read Thread 1 completed 755 loops. read Thread 2 completed 639 loops. fork Thread 9 completed 9 loops. fork Thread 10 completed 10 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 18037 loops. alloc Thread 16 completed 46293 loops. alloc Thread 17 completed 82560 loops. lock Thread 21 completed 4210008 loops. lock Thread 22 completed 2708008 loops. lock Thread 23 completed 6139723 loops. .. *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0xc4e39a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Coleburn, Randy Sent: Monday, January 13, 2014 3:14 PM To: Tony Hosking Cc: m3devel Subject: Re: [M3devel] CM3 crashing Tony: Yes, it is from the HEAD as of 17 December 2013. Using the @M3paranoidgc flag, On 32-bit Windows XP I get: C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest . *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x431755 = RefSanityCheck + 0x2c in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. On 64-bit Windows 7 I get: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest . *** *** runtime error: *** An array subscript was out of range. *** file "..\src\rw\FileRd.m3", line 83 *** *** *** runtime error: *** An enumeration or subrange value was out of range. *** file "..\src\runtime\common\RTType.m3", line 71 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Tony Hosking [mailto:hosking at cs.purdue.edu] Sent: Monday, January 13, 2014 3:03 PM To: Coleburn, Randy Cc: Peter McKinna; m3devel Subject: EXT:Re: [M3devel] CM3 crashing Is this from the head of source? Also, please run with @M3paranoidgc flag. On Jan 13, 2014, at 2:55 PM, Coleburn, Randy > wrote: I've also tested on 32-bit Windows XP and the thread test program crashes there also. C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386 C:\cm3\Sandbox\cm3\m3-libs>cm3 --version Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2013-12-17 17:28:52 configuration: C:\cm3\bin\cm3.cfg host: NT386 target: NT386 Here is the output: C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Coleburn, Randy Sent: Monday, January 13, 2014 2:17 PM To: Tony Hosking; Peter McKinna Cc: m3devel Subject: Re: [M3devel] CM3 crashing Tony et al: The thread test program fails for me on 64-bit Windows 7. I've listed output from a couple of runs below: C:\cm3\Sandbox>cm3 --version Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2014-01-07 06:55:14 configuration: C:\cm3\bin\cm3.cfg host: NT386 target: NT386 FIRST RUN: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\runtime\common\RTCollector.m3", line 418 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** ...this last assertion repeats until you press CTRL-C to abort the program... SECOND RUN: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) .... *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x776f22d2 *** Stack trace: FP PC Procedure --------- --------- ------------------------------- 0xa6f9a8 0xc6afeb SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3 0xa6f9d0 0x776f22d2 0xa6f9e8 0xc4a39b LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3 0xa6fa10 0xc299f8 GetChar + 0x28 in ..\src\rw\Rd.m3 0xa6fb48 0xc213a8 RApply + 0x1b8 in ..\src\Main.m3 0xa6fb84 0xc4b71b ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3 0xa6fb90 0x76db336a 0xa6fbd0 0x7770bf32 ......... ......... ... more frames ... Maybe this info will be useful in tracking down the problem. --Randy From: Tony Hosking [mailto:hosking at cs.purdue.edu] Sent: Monday, January 13, 2014 11:04 AM To: Peter McKinna Cc: m3devel Subject: EXT:Re: [M3devel] CM3 crashing Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?). In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem. As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC. Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning. Can someone confirm? If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed. On Jan 12, 2014, at 10:54 PM, Peter McKinna > wrote: Hey, I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped the second call to p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one. It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. Regards Peter On Mon, Jan 13, 2014 at 11:25 AM, > wrote: Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3). I figured there aren't actually implementations of Word.Xor, Word.Or, and Word.And as procedures but they get inlined somehow? BTW, any idea about what's wrong with pthreads? Do you think the issue is with the garbage collector or with the threads, off the top of your head? Tony Hosking writes: >Are you saying passing these used to work in PM3? >Sounds like a front-end bug. I=92m curious what changed to break it. > >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: > >>=20 >> The code is: >>=20 >> HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Xor), ab) >> | >> HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Or), ab) >> | >> HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, = >Word.And), ab) >>=20 >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20 > > > >Antony Hosking | Associate Professor | Computer Science | Purdue = >University >305 N. University Street | West Lafayette | IN 47907 | USA >Mobile +1 765 427 5484 > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Jan 14 00:22:12 2014 From: jay.krell at cornell.edu (Jay K) Date: Mon, 13 Jan 2014 23:22:12 +0000 Subject: [M3devel] CM3 crashing In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D3DF@ATLEX04-SRV.SCIRES.LOCAL> References: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D28A@ATLEX04-SRV.SCIRES.LOCAL>, <0BB8FA59C2932741A3A2941A8B9D8BFF9251D3DF@ATLEX04-SRV.SCIRES.LOCAL> Message-ID: I am a bit nervous about our use GetThreadContext in 32bit code on 64bit Windows per here: http://zachsaw.blogspot.com/2010/11/wow64-bug-getthreadcontext-may-return.html And disappointed. Cooperative suspend will fix this someday. - Jay From: rcolebur at SCIRES.COM To: hosking at cs.purdue.edu Date: Mon, 13 Jan 2014 21:02:52 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] CM3 crashing Tony: Here are a couple more runs on 64-bit Windows 7 that yield different results, albeit using the ?-verbose? option. These may give some more clues. I note that in both cases it appears a read thread and a fork thread get the same ID (0) which looks very suspicious. By my read of the code (WITH id = i * nPer + j), I think the 2nd fork thread should be ID=10, not ID=0. Oops, now I see the problem: there is a bug in the PutInt procedure. The test (IF c > 10) should read instead (IF c >= 10). I?ll commit a fix for that momentarily. But this fix just solves the output problem; it doesn?t affect the fact that the program is misbehaving and crashing. RUN #1: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc -verbose Writing file...done Creating read threads... read=0 read=1 read=2 done Creating fork threads... fork=9 fork=0 fork=11 done Creating alloc threads... alloc=15 alloc=16 alloc=17 done Creating lock threads... lock=21 lock=22 lock=23 done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0xc286e0 = Init + 0x79 in ..\src\rw\FileRd.m3 *** Stack trace: FP PC Procedure --------- --------- ------------------------------- 0x7cfd60 0xc6afeb SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3 0x7cfda8 0xc286e0 Init + 0x79 in ..\src\rw\FileRd.m3 0x7cfdd4 0xc2865d Open + 0x4d in ..\src\rw\FileRd.m3 0x7cff0c 0xc21350 RApply + 0x160 in ..\src\Main.m3 0x7cff48 0xc4b71b ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3 0x7cff54 0x76db336a 0x7cff94 0x7770bf32 ......... ......... ... more frames ... RUN #2: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe -verbose Writing file...done Creating read threads... read=0 read=1 read=2 done Creating fork threads... fork=9 fork=0 fork=11 done Creating alloc threads... alloc=15 alloc=16 alloc=17 done Creating lock threads... lock=21 lock=22 lock=23 done running...printing oldest/median age/newest ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 534 loops. read Thread 1 completed 606 loops. read Thread 2 completed 398 loops. fork Thread 9 completed 8 loops. fork Thread 10 completed 8 loops. fork Thread 11 completed 8 loops. alloc Thread 15 completed 18296 loops. alloc Thread 16 completed 44871 loops. alloc Thread 17 completed 79125 loops. lock Thread 21 completed 3845080 loops. lock Thread 22 completed 2537613 loops. lock Thread 23 completed 4506151 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 790 loops. read Thread 1 completed 786 loops. read Thread 2 completed 675 loops. fork Thread 9 completed 9 loops. fork Thread 10 completed 9 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 22091 loops. alloc Thread 16 completed 47532 loops. alloc Thread 17 completed 68901 loops. lock Thread 21 completed 4560731 loops. lock Thread 22 completed 3440795 loops. lock Thread 23 completed 6433538 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 725 loops. read Thread 1 completed 705 loops. read Thread 2 completed 617 loops. fork Thread 9 completed 10 loops. fork Thread 10 completed 9 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 18560 loops. alloc Thread 16 completed 44487 loops. alloc Thread 17 completed 87219 loops. lock Thread 21 completed 3769840 loops. lock Thread 22 completed 3037581 loops. lock Thread 23 completed 6097922 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 825 loops. read Thread 1 completed 802 loops. read Thread 2 completed 669 loops. fork Thread 9 completed 9 loops. fork Thread 10 completed 10 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 16877 loops. alloc Thread 16 completed 53277 loops. alloc Thread 17 completed 77986 loops. lock Thread 21 completed 4087218 loops. lock Thread 22 completed 3116357 loops. lock Thread 23 completed 6466214 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 684 loops. read Thread 1 completed 686 loops. read Thread 2 completed 648 loops. fork Thread 9 completed 10 loops. fork Thread 10 completed 9 loops. fork Thread 11 completed 10 loops. alloc Thread 15 completed 19717 loops. alloc Thread 16 completed 47797 loops. alloc Thread 17 completed 78110 loops. lock Thread 21 completed 4580435 loops. lock Thread 22 completed 2974247 loops. lock Thread 23 completed 5843850 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 748 loops. read Thread 1 completed 755 loops. read Thread 2 completed 639 loops. fork Thread 9 completed 9 loops. fork Thread 10 completed 10 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 18037 loops. alloc Thread 16 completed 46293 loops. alloc Thread 17 completed 82560 loops. lock Thread 21 completed 4210008 loops. lock Thread 22 completed 2708008 loops. lock Thread 23 completed 6139723 loops. .. *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0xc4e39a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Coleburn, Randy Sent: Monday, January 13, 2014 3:14 PM To: Tony Hosking Cc: m3devel Subject: Re: [M3devel] CM3 crashing Tony: Yes, it is from the HEAD as of 17 December 2013. Using the @M3paranoidgc flag, On 32-bit Windows XP I get: C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest . *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x431755 = RefSanityCheck + 0x2c in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. On 64-bit Windows 7 I get: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest . *** *** runtime error: *** An array subscript was out of range. *** file "..\src\rw\FileRd.m3", line 83 *** *** *** runtime error: *** An enumeration or subrange value was out of range. *** file "..\src\runtime\common\RTType.m3", line 71 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Tony Hosking [mailto:hosking at cs.purdue.edu] Sent: Monday, January 13, 2014 3:03 PM To: Coleburn, Randy Cc: Peter McKinna; m3devel Subject: EXT:Re: [M3devel] CM3 crashing Is this from the head of source? Also, please run with @M3paranoidgc flag. On Jan 13, 2014, at 2:55 PM, Coleburn, Randy wrote: I?ve also tested on 32-bit Windows XP and the thread test program crashes there also. C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386 C:\cm3\Sandbox\cm3\m3-libs>cm3 --version Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2013-12-17 17:28:52 configuration: C:\cm3\bin\cm3.cfg host: NT386 target: NT386 Here is the output: C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Coleburn, Randy Sent: Monday, January 13, 2014 2:17 PM To: Tony Hosking; Peter McKinna Cc: m3devel Subject: Re: [M3devel] CM3 crashing Tony et al: The thread test program fails for me on 64-bit Windows 7. I?ve listed output from a couple of runs below: C:\cm3\Sandbox>cm3 --version Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2014-01-07 06:55:14 configuration: C:\cm3\bin\cm3.cfg host: NT386 target: NT386 FIRST RUN: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\runtime\common\RTCollector.m3", line 418 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** ?this last assertion repeats until you press CTRL-C to abort the program? SECOND RUN: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) .... *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x776f22d2 *** Stack trace: FP PC Procedure --------- --------- ------------------------------- 0xa6f9a8 0xc6afeb SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3 0xa6f9d0 0x776f22d2 0xa6f9e8 0xc4a39b LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3 0xa6fa10 0xc299f8 GetChar + 0x28 in ..\src\rw\Rd.m3 0xa6fb48 0xc213a8 RApply + 0x1b8 in ..\src\Main.m3 0xa6fb84 0xc4b71b ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3 0xa6fb90 0x76db336a 0xa6fbd0 0x7770bf32 ......... ......... ... more frames ... Maybe this info will be useful in tracking down the problem. --Randy From: Tony Hosking [mailto:hosking at cs.purdue.edu] Sent: Monday, January 13, 2014 11:04 AM To: Peter McKinna Cc: m3devel Subject: EXT:Re: [M3devel] CM3 crashing Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?). In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem. As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC. Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning. Can someone confirm? If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed. On Jan 12, 2014, at 10:54 PM, Peter McKinna wrote: Hey, I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped the second call to p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one. It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. Regards Peter On Mon, Jan 13, 2014 at 11:25 AM, wrote: Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3). I figured there aren't actually implementations of Word.Xor, Word.Or, and Word.And as procedures but they get inlined somehow? BTW, any idea about what's wrong with pthreads? Do you think the issue is with the garbage collector or with the threads, off the top of your head? Tony Hosking writes: >Are you saying passing these used to work in PM3? >Sounds like a front-end bug. I=92m curious what changed to break it. > >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: > >>=20 >> The code is: >>=20 >> HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Xor), ab) >> | >> HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Or), ab) >> | >> HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, = >Word.And), ab) >>=20 >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20 > > > >Antony Hosking | Associate Professor | Computer Science | Purdue = >University >305 N. University Street | West Lafayette | IN 47907 | USA >Mobile +1 765 427 5484 > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcolebur at SCIRES.COM Tue Jan 14 01:11:07 2014 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Tue, 14 Jan 2014 00:11:07 +0000 Subject: [M3devel] CM3 crashing Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D74F@ATLEX04-SRV.SCIRES.LOCAL> Jay: The article you reference is dated 13 Nov 2010. Do you know whether Microsoft has done anything about this problem over the past 3 years? Also, the thread test program crashes on 32-bit XP, so there must be some additional bug(s) somewhere besides the WOW64 bug wrt GetThreadContext. --Randy From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Monday, January 13, 2014 6:22 PM To: Coleburn, Randy; Tony Cc: m3devel Subject: EXT:RE: [M3devel] CM3 crashing I am a bit nervous about our use GetThreadContext in 32bit code on 64bit Windows per here: http://zachsaw.blogspot.com/2010/11/wow64-bug-getthreadcontext-may-return.html And disappointed. Cooperative suspend will fix this someday. - Jay ________________________________ From: rcolebur at SCIRES.COM To: hosking at cs.purdue.edu Date: Mon, 13 Jan 2014 21:02:52 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] CM3 crashing Tony: Here are a couple more runs on 64-bit Windows 7 that yield different results, albeit using the "-verbose" option. These may give some more clues. I note that in both cases it appears a read thread and a fork thread get the same ID (0) which looks very suspicious. By my read of the code (WITH id = i * nPer + j), I think the 2nd fork thread should be ID=10, not ID=0. Oops, now I see the problem: there is a bug in the PutInt procedure. The test (IF c > 10) should read instead (IF c >= 10). I'll commit a fix for that momentarily. But this fix just solves the output problem; it doesn't affect the fact that the program is misbehaving and crashing. RUN #1: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc -verbose Writing file...done Creating read threads... read=0 read=1 read=2 done Creating fork threads... fork=9 fork=0 fork=11 done Creating alloc threads... alloc=15 alloc=16 alloc=17 done Creating lock threads... lock=21 lock=22 lock=23 done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0xc286e0 = Init + 0x79 in ..\src\rw\FileRd.m3 *** Stack trace: FP PC Procedure --------- --------- ------------------------------- 0x7cfd60 0xc6afeb SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3 0x7cfda8 0xc286e0 Init + 0x79 in ..\src\rw\FileRd.m3 0x7cfdd4 0xc2865d Open + 0x4d in ..\src\rw\FileRd.m3 0x7cff0c 0xc21350 RApply + 0x160 in ..\src\Main.m3 0x7cff48 0xc4b71b ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3 0x7cff54 0x76db336a 0x7cff94 0x7770bf32 ......... ......... ... more frames ... RUN #2: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe -verbose Writing file...done Creating read threads... read=0 read=1 read=2 done Creating fork threads... fork=9 fork=0 fork=11 done Creating alloc threads... alloc=15 alloc=16 alloc=17 done Creating lock threads... lock=21 lock=22 lock=23 done running...printing oldest/median age/newest ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 534 loops. read Thread 1 completed 606 loops. read Thread 2 completed 398 loops. fork Thread 9 completed 8 loops. fork Thread 10 completed 8 loops. fork Thread 11 completed 8 loops. alloc Thread 15 completed 18296 loops. alloc Thread 16 completed 44871 loops. alloc Thread 17 completed 79125 loops. lock Thread 21 completed 3845080 loops. lock Thread 22 completed 2537613 loops. lock Thread 23 completed 4506151 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 790 loops. read Thread 1 completed 786 loops. read Thread 2 completed 675 loops. fork Thread 9 completed 9 loops. fork Thread 10 completed 9 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 22091 loops. alloc Thread 16 completed 47532 loops. alloc Thread 17 completed 68901 loops. lock Thread 21 completed 4560731 loops. lock Thread 22 completed 3440795 loops. lock Thread 23 completed 6433538 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 725 loops. read Thread 1 completed 705 loops. read Thread 2 completed 617 loops. fork Thread 9 completed 10 loops. fork Thread 10 completed 9 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 18560 loops. alloc Thread 16 completed 44487 loops. alloc Thread 17 completed 87219 loops. lock Thread 21 completed 3769840 loops. lock Thread 22 completed 3037581 loops. lock Thread 23 completed 6097922 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 825 loops. read Thread 1 completed 802 loops. read Thread 2 completed 669 loops. fork Thread 9 completed 9 loops. fork Thread 10 completed 10 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 16877 loops. alloc Thread 16 completed 53277 loops. alloc Thread 17 completed 77986 loops. lock Thread 21 completed 4087218 loops. lock Thread 22 completed 3116357 loops. lock Thread 23 completed 6466214 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 684 loops. read Thread 1 completed 686 loops. read Thread 2 completed 648 loops. fork Thread 9 completed 10 loops. fork Thread 10 completed 9 loops. fork Thread 11 completed 10 loops. alloc Thread 15 completed 19717 loops. alloc Thread 16 completed 47797 loops. alloc Thread 17 completed 78110 loops. lock Thread 21 completed 4580435 loops. lock Thread 22 completed 2974247 loops. lock Thread 23 completed 5843850 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 748 loops. read Thread 1 completed 755 loops. read Thread 2 completed 639 loops. fork Thread 9 completed 9 loops. fork Thread 10 completed 10 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 18037 loops. alloc Thread 16 completed 46293 loops. alloc Thread 17 completed 82560 loops. lock Thread 21 completed 4210008 loops. lock Thread 22 completed 2708008 loops. lock Thread 23 completed 6139723 loops. .. *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0xc4e39a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Coleburn, Randy Sent: Monday, January 13, 2014 3:14 PM To: Tony Hosking Cc: m3devel Subject: Re: [M3devel] CM3 crashing Tony: Yes, it is from the HEAD as of 17 December 2013. Using the @M3paranoidgc flag, On 32-bit Windows XP I get: C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest . *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x431755 = RefSanityCheck + 0x2c in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. On 64-bit Windows 7 I get: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest . *** *** runtime error: *** An array subscript was out of range. *** file "..\src\rw\FileRd.m3", line 83 *** *** *** runtime error: *** An enumeration or subrange value was out of range. *** file "..\src\runtime\common\RTType.m3", line 71 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Tony Hosking [mailto:hosking at cs.purdue.edu] Sent: Monday, January 13, 2014 3:03 PM To: Coleburn, Randy Cc: Peter McKinna; m3devel Subject: EXT:Re: [M3devel] CM3 crashing Is this from the head of source? Also, please run with @M3paranoidgc flag. On Jan 13, 2014, at 2:55 PM, Coleburn, Randy > wrote: I've also tested on 32-bit Windows XP and the thread test program crashes there also. C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386 C:\cm3\Sandbox\cm3\m3-libs>cm3 --version Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2013-12-17 17:28:52 configuration: C:\cm3\bin\cm3.cfg host: NT386 target: NT386 Here is the output: C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Coleburn, Randy Sent: Monday, January 13, 2014 2:17 PM To: Tony Hosking; Peter McKinna Cc: m3devel Subject: Re: [M3devel] CM3 crashing Tony et al: The thread test program fails for me on 64-bit Windows 7. I've listed output from a couple of runs below: C:\cm3\Sandbox>cm3 --version Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2014-01-07 06:55:14 configuration: C:\cm3\bin\cm3.cfg host: NT386 target: NT386 FIRST RUN: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\runtime\common\RTCollector.m3", line 418 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** ...this last assertion repeats until you press CTRL-C to abort the program... SECOND RUN: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) .... *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x776f22d2 *** Stack trace: FP PC Procedure --------- --------- ------------------------------- 0xa6f9a8 0xc6afeb SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3 0xa6f9d0 0x776f22d2 0xa6f9e8 0xc4a39b LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3 0xa6fa10 0xc299f8 GetChar + 0x28 in ..\src\rw\Rd.m3 0xa6fb48 0xc213a8 RApply + 0x1b8 in ..\src\Main.m3 0xa6fb84 0xc4b71b ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3 0xa6fb90 0x76db336a 0xa6fbd0 0x7770bf32 ......... ......... ... more frames ... Maybe this info will be useful in tracking down the problem. --Randy From: Tony Hosking [mailto:hosking at cs.purdue.edu] Sent: Monday, January 13, 2014 11:04 AM To: Peter McKinna Cc: m3devel Subject: EXT:Re: [M3devel] CM3 crashing Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?). In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem. As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC. Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning. Can someone confirm? If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed. On Jan 12, 2014, at 10:54 PM, Peter McKinna > wrote: Hey, I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped the second call to p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one. It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. Regards Peter On Mon, Jan 13, 2014 at 11:25 AM, > wrote: Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3). I figured there aren't actually implementations of Word.Xor, Word.Or, and Word.And as procedures but they get inlined somehow? BTW, any idea about what's wrong with pthreads? Do you think the issue is with the garbage collector or with the threads, off the top of your head? Tony Hosking writes: >Are you saying passing these used to work in PM3? >Sounds like a front-end bug. I=92m curious what changed to break it. > >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: > >>=20 >> The code is: >>=20 >> HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Xor), ab) >> | >> HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Or), ab) >> | >> HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, = >Word.And), ab) >>=20 >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20 > > > >Antony Hosking | Associate Professor | Computer Science | Purdue = >University >305 N. University Street | West Lafayette | IN 47907 | USA >Mobile +1 765 427 5484 > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Jan 14 01:21:49 2014 From: jay.krell at cornell.edu (Jay K) Date: Tue, 14 Jan 2014 00:21:49 +0000 Subject: [M3devel] CM3 crashing In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D74F@ATLEX04-SRV.SCIRES.LOCAL> References: <0BB8FA59C2932741A3A2941A8B9D8BFF9251D74F@ATLEX04-SRV.SCIRES.LOCAL> Message-ID: I don't know. We can try writing a stress test for it. It looks like one is available. Agreed -- work out the bugs in the native Windows and Posix systems. - Jay From: rcolebur at SCIRES.COM To: jay.krell at cornell.edu; hosking at cs.purdue.edu Date: Tue, 14 Jan 2014 00:11:07 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] CM3 crashing Jay: The article you reference is dated 13 Nov 2010. Do you know whether Microsoft has done anything about this problem over the past 3 years? Also, the thread test program crashes on 32-bit XP, so there must be some additional bug(s) somewhere besides the WOW64 bug wrt GetThreadContext. --Randy From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Monday, January 13, 2014 6:22 PM To: Coleburn, Randy; Tony Cc: m3devel Subject: EXT:RE: [M3devel] CM3 crashing I am a bit nervous about our use GetThreadContext in 32bit code on 64bit Windows per here: http://zachsaw.blogspot.com/2010/11/wow64-bug-getthreadcontext-may-return.html And disappointed. Cooperative suspend will fix this someday. - Jay From: rcolebur at SCIRES.COM To: hosking at cs.purdue.edu Date: Mon, 13 Jan 2014 21:02:52 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] CM3 crashing Tony: Here are a couple more runs on 64-bit Windows 7 that yield different results, albeit using the ?-verbose? option. These may give some more clues. I note that in both cases it appears a read thread and a fork thread get the same ID (0) which looks very suspicious. By my read of the code (WITH id = i * nPer + j), I think the 2nd fork thread should be ID=10, not ID=0. Oops, now I see the problem: there is a bug in the PutInt procedure. The test (IF c > 10) should read instead (IF c >= 10). I?ll commit a fix for that momentarily. But this fix just solves the output problem; it doesn?t affect the fact that the program is misbehaving and crashing. RUN #1: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc -verbose Writing file...done Creating read threads... read=0 read=1 read=2 done Creating fork threads... fork=9 fork=0 fork=11 done Creating alloc threads... alloc=15 alloc=16 alloc=17 done Creating lock threads... lock=21 lock=22 lock=23 done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0xc286e0 = Init + 0x79 in ..\src\rw\FileRd.m3 *** Stack trace: FP PC Procedure --------- --------- ------------------------------- 0x7cfd60 0xc6afeb SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3 0x7cfda8 0xc286e0 Init + 0x79 in ..\src\rw\FileRd.m3 0x7cfdd4 0xc2865d Open + 0x4d in ..\src\rw\FileRd.m3 0x7cff0c 0xc21350 RApply + 0x160 in ..\src\Main.m3 0x7cff48 0xc4b71b ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3 0x7cff54 0x76db336a 0x7cff94 0x7770bf32 ......... ......... ... more frames ... RUN #2: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe -verbose Writing file...done Creating read threads... read=0 read=1 read=2 done Creating fork threads... fork=9 fork=0 fork=11 done Creating alloc threads... alloc=15 alloc=16 alloc=17 done Creating lock threads... lock=21 lock=22 lock=23 done running...printing oldest/median age/newest ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 534 loops. read Thread 1 completed 606 loops. read Thread 2 completed 398 loops. fork Thread 9 completed 8 loops. fork Thread 10 completed 8 loops. fork Thread 11 completed 8 loops. alloc Thread 15 completed 18296 loops. alloc Thread 16 completed 44871 loops. alloc Thread 17 completed 79125 loops. lock Thread 21 completed 3845080 loops. lock Thread 22 completed 2537613 loops. lock Thread 23 completed 4506151 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 790 loops. read Thread 1 completed 786 loops. read Thread 2 completed 675 loops. fork Thread 9 completed 9 loops. fork Thread 10 completed 9 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 22091 loops. alloc Thread 16 completed 47532 loops. alloc Thread 17 completed 68901 loops. lock Thread 21 completed 4560731 loops. lock Thread 22 completed 3440795 loops. lock Thread 23 completed 6433538 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 725 loops. read Thread 1 completed 705 loops. read Thread 2 completed 617 loops. fork Thread 9 completed 10 loops. fork Thread 10 completed 9 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 18560 loops. alloc Thread 16 completed 44487 loops. alloc Thread 17 completed 87219 loops. lock Thread 21 completed 3769840 loops. lock Thread 22 completed 3037581 loops. lock Thread 23 completed 6097922 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 825 loops. read Thread 1 completed 802 loops. read Thread 2 completed 669 loops. fork Thread 9 completed 9 loops. fork Thread 10 completed 10 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 16877 loops. alloc Thread 16 completed 53277 loops. alloc Thread 17 completed 77986 loops. lock Thread 21 completed 4087218 loops. lock Thread 22 completed 3116357 loops. lock Thread 23 completed 6466214 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 684 loops. read Thread 1 completed 686 loops. read Thread 2 completed 648 loops. fork Thread 9 completed 10 loops. fork Thread 10 completed 9 loops. fork Thread 11 completed 10 loops. alloc Thread 15 completed 19717 loops. alloc Thread 16 completed 47797 loops. alloc Thread 17 completed 78110 loops. lock Thread 21 completed 4580435 loops. lock Thread 22 completed 2974247 loops. lock Thread 23 completed 5843850 loops. ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) read Thread 0 completed 748 loops. read Thread 1 completed 755 loops. read Thread 2 completed 639 loops. fork Thread 9 completed 9 loops. fork Thread 10 completed 10 loops. fork Thread 11 completed 9 loops. alloc Thread 15 completed 18037 loops. alloc Thread 16 completed 46293 loops. alloc Thread 17 completed 82560 loops. lock Thread 21 completed 4210008 loops. lock Thread 22 completed 2708008 loops. lock Thread 23 completed 6139723 loops. .. *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0xc4e39a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Coleburn, Randy Sent: Monday, January 13, 2014 3:14 PM To: Tony Hosking Cc: m3devel Subject: Re: [M3devel] CM3 crashing Tony: Yes, it is from the HEAD as of 17 December 2013. Using the @M3paranoidgc flag, On 32-bit Windows XP I get: C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest . *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x431755 = RefSanityCheck + 0x2c in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. On 64-bit Windows 7 I get: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe @M3paranoidgc Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest . *** *** runtime error: *** An array subscript was out of range. *** file "..\src\rw\FileRd.m3", line 83 *** *** *** runtime error: *** An enumeration or subrange value was out of range. *** file "..\src\runtime\common\RTType.m3", line 71 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Tony Hosking [mailto:hosking at cs.purdue.edu] Sent: Monday, January 13, 2014 3:03 PM To: Coleburn, Randy Cc: Peter McKinna; m3devel Subject: EXT:Re: [M3devel] CM3 crashing Is this from the head of source? Also, please run with @M3paranoidgc flag. On Jan 13, 2014, at 2:55 PM, Coleburn, Randy wrote: I?ve also tested on 32-bit Windows XP and the thread test program crashes there also. C:\cm3\Sandbox\cm3\m3-libs>set CM3_TARGET=NT386 C:\cm3\Sandbox\cm3\m3-libs>cm3 --version Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2013-12-17 17:28:52 configuration: C:\cm3\bin\cm3.cfg host: NT386 target: NT386 Here is the output: C:\cm3\Sandbox\cm3\m3-libs\m3core\tests\thread\NT386>threadtest.exe --help Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x42e34a = Move + 0x50 in ..\src\runtime\common\RTCollector.m3 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** This last assert repeats indefinitely until you press CTRL-C. --Randy From: Coleburn, Randy Sent: Monday, January 13, 2014 2:17 PM To: Tony Hosking; Peter McKinna Cc: m3devel Subject: Re: [M3devel] CM3 crashing Tony et al: The thread test program fails for me on 64-bit Windows 7. I?ve listed output from a couple of runs below: C:\cm3\Sandbox>cm3 --version Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2014-01-07 06:55:14 configuration: C:\cm3\bin\cm3.cfg host: NT386 target: NT386 FIRST RUN: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** Attempt to reference an illegal memory location. *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\runtime\common\RTCollector.m3", line 418 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** *** *** runtime error: *** <*ASSERT*> failed. *** file "..\src\thread\WIN32\ThreadWin32.m3", line 823 *** ?this last assertion repeats until you press CTRL-C to abort the program? SECOND RUN: C:\cm3\Sandbox\m3-libs\m3core\tests\thread\NT386>threadtest.exe /help Writing file...done Creating read threads...done Creating fork threads...done Creating alloc threads...done Creating lock threads...done running...printing oldest/median age/newest ..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock 0/0/0) .... *** *** runtime error: *** Attempt to reference an illegal memory location. *** pc = 0x776f22d2 *** Stack trace: FP PC Procedure --------- --------- ------------------------------- 0xa6f9a8 0xc6afeb SystemError + 0x64 in ..\src\runtime\NT386\RTSignal.m3 0xa6f9d0 0x776f22d2 0xa6f9e8 0xc4a39b LockMutex + 0x4f in ..\src\thread\WIN32\ThreadWin32.m3 0xa6fa10 0xc299f8 GetChar + 0x28 in ..\src\rw\Rd.m3 0xa6fb48 0xc213a8 RApply + 0x1b8 in ..\src\Main.m3 0xa6fb84 0xc4b71b ThreadBase + 0x255 in ..\src\thread\WIN32\ThreadWin32.m3 0xa6fb90 0x76db336a 0xa6fbd0 0x7770bf32 ......... ......... ... more frames ... Maybe this info will be useful in tracking down the problem. --Randy From: Tony Hosking [mailto:hosking at cs.purdue.edu] Sent: Monday, January 13, 2014 11:04 AM To: Peter McKinna Cc: m3devel Subject: EXT:Re: [M3devel] CM3 crashing Let us assume that the user-level threads are functioning properly w.r.to GC (can someone confirm?). In which case, it would be good to have as many eyes as possible take a look at the differences between ThreadPosixC.c (ProcessContext) and ThreadPThreadC.c (ProcessLive) to see if we can spot a problem. As I understand it, the crash occurs even for non-concurrent (@M3noincremental) and non-generational (@M3nogenerational) GC. Combined with my assumption that user threads work fine, it would seem to point the finger at stack scanning. Can someone confirm? If the failure is *only* with concurrent or generational collection then we might suspect unsafe code (perhaps newly introduced?) messing with heap references without keeping the collector informed. On Jan 12, 2014, at 10:54 PM, Peter McKinna wrote: Hey, I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped the second call to p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one. It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. Regards Peter On Mon, Jan 13, 2014 at 11:25 AM, wrote: Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3). I figured there aren't actually implementations of Word.Xor, Word.Or, and Word.And as procedures but they get inlined somehow? BTW, any idea about what's wrong with pthreads? Do you think the issue is with the garbage collector or with the threads, off the top of your head? Tony Hosking writes: >Are you saying passing these used to work in PM3? >Sounds like a front-end bug. I=92m curious what changed to break it. > >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: > >>=20 >> The code is: >>=20 >> HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Xor), ab) >> | >> HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Or), ab) >> | >> HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, = >Word.And), ab) >>=20 >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20 > > > >Antony Hosking | Associate Professor | Computer Science | Purdue = >University >305 N. University Street | West Lafayette | IN 47907 | USA >Mobile +1 765 427 5484 > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.mckinna at gmail.com Mon Jan 13 04:54:08 2014 From: peter.mckinna at gmail.com (Peter McKinna) Date: Mon, 13 Jan 2014 14:54:08 +1100 Subject: [M3devel] CM3 crashing In-Reply-To: <20140113002539.EC4EE1A2080@async.async.caltech.edu> References: <20140112175317.9F0031A2080@async.async.caltech.edu> <20140112175830.C51881A2080@async.async.caltech.edu> <20140113002539.EC4EE1A2080@async.async.caltech.edu> Message-ID: Hey, I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped the second call to p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one. It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. Regards Peter On Mon, Jan 13, 2014 at 11:25 AM, wrote: > Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a > reason to switch to CM3). > > I figured there aren't actually implementations of Word.Xor, Word.Or, > and Word.And as procedures but they get inlined somehow? > > BTW, any idea about what's wrong with pthreads? Do you think the issue > is with the garbage collector or with the threads, off the top of your > head? > > Tony Hosking writes: > >Are you saying passing these used to work in PM3? > >Sounds like a front-end bug. I=92m curious what changed to break it. > > > >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: > > > >>=20 > >> The code is: > >>=20 > >> HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, = > >Word.Xor), ab) > >> | > >> HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, = > >Word.Or), ab) > >> | > >> HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, = > >Word.And), ab) > >>=20 > >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20 > > > > > > > >Antony Hosking | Associate Professor | Computer Science | Purdue = > >University > >305 N. University Street | West Lafayette | IN 47907 | USA > >Mobile +1 765 427 5484 > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jan 15 06:01:16 2014 From: jay.krell at cornell.edu (Jay K) Date: Wed, 15 Jan 2014 05:01:16 +0000 Subject: [M3devel] CM3 crashing In-Reply-To: References: <20140112175317.9F0031A2080@async.async.caltech.edu>, <20140112175830.C51881A2080@async.async.caltech.edu>, , <20140113002539.EC4EE1A2080@async.async.caltech.edu>, Message-ID: Here is my failed attempt to demonstrate the wow64 bug. Make sense? Repros for anyone? I only tried Windows 8 so far. #undef NDEBUG #include #include #include void* _AddressOfReturnAddress(void); volatile size_t expected; #define PAGE 4096 __declspec(noinline) void X(void) { expected = (size_t)_AddressOfReturnAddress(); Sleep(1); expected = 0; } __declspec(noinline) void F1(void) { volatile char a[PAGE * 8]; X(); } __declspec(noinline) void F2(void) { volatile char a[PAGE * 4]; X(); } __declspec(noinline) unsigned long __stdcall Thread(PVOID parameter) { while (1) { F1(); F2(); } return 0; } int __cdecl wmain() { unsigned __int64 count = 0; HANDLE thread = CreateThread(0, 0, Thread, 0, 0, 0); CONTEXT context; ZeroMemory(&context, sizeof(context)); context.ContextFlags = CONTEXT_CONTROL; //while (count++ < 1000000) while (1) { SuspendThread(thread); GetThreadContext(thread, &context); assert(expected == 0 || (context.Esp && context.Esp < expected)); ResumeThread(thread); } } - Jay Date: Mon, 13 Jan 2014 14:54:08 +1100 From: peter.mckinna at gmail.com To: mika at async.caltech.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] CM3 crashing Hey, I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped the second call to p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one. It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. Regards Peter On Mon, Jan 13, 2014 at 11:25 AM, wrote: Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3). I figured there aren't actually implementations of Word.Xor, Word.Or, and Word.And as procedures but they get inlined somehow? BTW, any idea about what's wrong with pthreads? Do you think the issue is with the garbage collector or with the threads, off the top of your head? Tony Hosking writes: >Are you saying passing these used to work in PM3? >Sounds like a front-end bug. I=92m curious what changed to break it. > >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: > >>=20 >> The code is: >>=20 >> HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Xor), ab) >> | >> HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Or), ab) >> | >> HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, = >Word.And), ab) >>=20 >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20 > > > >Antony Hosking | Associate Professor | Computer Science | Purdue = >University >305 N. University Street | West Lafayette | IN 47907 | USA >Mobile +1 765 427 5484 > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jan 15 07:48:30 2014 From: jay.krell at cornell.edu (Jay K) Date: Wed, 15 Jan 2014 06:48:30 +0000 Subject: [M3devel] CM3 crashing In-Reply-To: References: <20140112175317.9F0031A2080@async.async.caltech.edu>, , <20140112175830.C51881A2080@async.async.caltech.edu>, , , , <20140113002539.EC4EE1A2080@async.async.caltech.edu>, , , Message-ID: ok, I augmented the test a bit and commited it to CVS and I can see it fail now. Does the test make sense to people? Thanks, - Jay From: jay.krell at cornell.edu To: peter.mckinna at gmail.com; mika at async.caltech.edu Date: Wed, 15 Jan 2014 05:01:16 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] CM3 crashing Here is my failed attempt to demonstrate the wow64 bug. Make sense? Repros for anyone? I only tried Windows 8 so far. #undef NDEBUG #include #include #include void* _AddressOfReturnAddress(void); volatile size_t expected; #define PAGE 4096 __declspec(noinline) void X(void) { expected = (size_t)_AddressOfReturnAddress(); Sleep(1); expected = 0; } __declspec(noinline) void F1(void) { volatile char a[PAGE * 8]; X(); } __declspec(noinline) void F2(void) { volatile char a[PAGE * 4]; X(); } __declspec(noinline) unsigned long __stdcall Thread(PVOID parameter) { while (1) { F1(); F2(); } return 0; } int __cdecl wmain() { unsigned __int64 count = 0; HANDLE thread = CreateThread(0, 0, Thread, 0, 0, 0); CONTEXT context; ZeroMemory(&context, sizeof(context)); context.ContextFlags = CONTEXT_CONTROL; //while (count++ < 1000000) while (1) { SuspendThread(thread); GetThreadContext(thread, &context); assert(expected == 0 || (context.Esp && context.Esp < expected)); ResumeThread(thread); } } - Jay Date: Mon, 13 Jan 2014 14:54:08 +1100 From: peter.mckinna at gmail.com To: mika at async.caltech.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] CM3 crashing Hey, I was trying to get a handle on that problem last year. The threadtest program is really a stress tester of the collector/allocator with pthreads. If you run it with just the tests read and alloc you pretty much always get a crash. If you run them with paranoidgc it will crash in the heap checker. Tony thought it was a clear cut problem of the roots of some ref not being found on a thread stack. I mucked around with code to get some output and the stacks looked ok to me but I could be wrong. All stacks are checked whilst the threads are blocked in a signal handler, the design of which looks fine as far as I can tell. This test is characterised by some slow threads (the read threads) and a bunch of fast threads (the alloc threads). Even if you modify the test to have only one read thread the problem occurs. I have had misgivings about mixing signals and threads having been bitten many years ago, but really this is the only way the collector can get its raw refs to check. My guess is its a subtle timing or lock problem maybe a lurking bug in the collector itself. One thing I noticed is that in ThreadPThread__ProcessStopped the second call to p(context, ((char *)context) + sizeof(ucontext_t)); according to the comment is to process the registers. But the registers should already be on the stack and anyway this call is a partial duplicate of the previous one. It would be good to raise the priority on this problem. Trust in the collector has always been at the heart of m3 programs. Regards Peter On Mon, Jan 13, 2014 at 11:25 AM, wrote: Yes it works in PM3 (still, since I use PM3 on FreeBSD4, never saw a reason to switch to CM3). I figured there aren't actually implementations of Word.Xor, Word.Or, and Word.And as procedures but they get inlined somehow? BTW, any idea about what's wrong with pthreads? Do you think the issue is with the garbage collector or with the threads, off the top of your head? Tony Hosking writes: >Are you saying passing these used to work in PM3? >Sounds like a front-end bug. I=92m curious what changed to break it. > >On Jan 12, 2014, at 12:58 PM, mika at async.caltech.edu wrote: > >>=20 >> The code is: >>=20 >> HIntExpr.Xor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Xor), ab) >> | >> HIntExpr.Bor =3D> RETURN NewConst(CBitwise(av, bv, = >Word.Or), ab) >> | >> HIntExpr.Band =3D> RETURN NewConst(CBitwise(av, bv, = >Word.And), ab) >>=20 >> I guess it doesn't like passing Word.Xor, Word.Or, and Word.And ...=20 > > > >Antony Hosking | Associate Professor | Computer Science | Purdue = >University >305 N. University Street | West Lafayette | IN 47907 | USA >Mobile +1 765 427 5484 > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sat Jan 18 07:22:53 2014 From: jay.krell at cornell.edu (Jay K) Date: Sat, 18 Jan 2014 06:22:53 +0000 Subject: [M3devel] Win32 SuspendThread? Message-ID: This program also doesn't behave as expected, native, nothing to do with wow64. Anyone else please confirm: 1) my expectations -- it should never print anything 2) their importance -- garbage collector depends on it 3) their not being met -- stuff gets printed This is in the CVS repository, scratch/wow64stack/sync2.cpp I am following up further. Maybe we should get cooperative suspend really going? Thank you. - Jay #include #include volatile long value; unsigned long __stdcall Thread(PVOID parameter) { while (1) InterlockedIncrement(&value); return 0; } int __cdecl main() { HANDLE thread = CreateThread(0, 0, Thread, 0, 0, 0); UINT i = 0; while (1) { i += 1; if (SuspendThread(thread) == (DWORD)-1) { printf("suspend failed %X\n", GetLastError()); Sleep(1); continue; } volatile long a = value; volatile long b = value; if (a != b) { printf("%d %d %d %d\n", i, a, b, b - a); } ResumeThread(thread); } } -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcolebur at SCIRES.COM Sat Jan 18 08:11:45 2014 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Sat, 18 Jan 2014 07:11:45 +0000 Subject: [M3devel] Win32 SuspendThread? Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL> Jay: I'm very concerned about the threading not working properly on both 32-bit and 64-bit Windows. The Thread Test program crashes for me on both platforms. I haven't tried your new test program yet. --Randy From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Saturday, January 18, 2014 1:23 AM To: m3devel Subject: EXT:[M3devel] Win32 SuspendThread? This program also doesn't behave as expected, native, nothing to do with wow64. Anyone else please confirm: 1) my expectations -- it should never print anything 2) their importance -- garbage collector depends on it 3) their not being met -- stuff gets printed This is in the CVS repository, scratch/wow64stack/sync2.cpp I am following up further. Maybe we should get cooperative suspend really going? Thank you. - Jay #include #include volatile long value; unsigned long __stdcall Thread(PVOID parameter) { while (1) InterlockedIncrement(&value); return 0; } int __cdecl main() { HANDLE thread = CreateThread(0, 0, Thread, 0, 0, 0); UINT i = 0; while (1) { i += 1; if (SuspendThread(thread) == (DWORD)-1) { printf("suspend failed %X\n", GetLastError()); Sleep(1); continue; } volatile long a = value; volatile long b = value; if (a != b) { printf("%d %d %d %d\n", i, a, b, b - a); } ResumeThread(thread); } } -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Sat Jan 18 09:24:11 2014 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sat, 18 Jan 2014 09:24:11 +0100 Subject: [M3devel] Cooperative suspend? Message-ID: <8BC1682A-3760-44D0-BA67-3A0F3EA10E27@m3w.org> Jay, What exactly do you mean? Isn?t it cooperative already? Thread stopping the world informs others about its intent, and they suspend themselves on a semaphore. Windows implementation of threading does forced suspend? dd -- Dragi?a Duri? dragisha at m3w.org -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Sun Jan 19 04:40:06 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sat, 18 Jan 2014 19:40:06 -0800 Subject: [M3devel] how is this possible? Message-ID: <20140119034006.34F511A208F@async.async.caltech.edu> Any hypothesis is welcome... (311)truffles:~/t/btc/hparse/src>../AMD64_LINUX/hparse and.scm *** *** runtime error: *** An explicit or implicit NARROW() operation failed. *** file "../AMD64_LINUX/BDDSchemeStubs.m3", line 619 *** Abort (312)truffles:~/t/btc/hparse/src> 613 PROCEDURE ToModula_BDD_T(x : SchemeObject.T) : (BDD.T) RAISES { Scheme.E } = 614 BEGIN 615 IF x # NIL AND ISTYPE(x, REF BDD.T) THEN RETURN NARROW(x,REF BDD.T)^ END; 616 IF NOT ISTYPE(x,BDD.T) THEN 617 RAISE Scheme.E("Not of type BDD.T : " & SchemeUtils.Stringify(x)) 618 END; 619 RETURN x 620 END ToModula_BDD_T; The object is of type BDD.Root (I checked the TYPECODE), which is declared thus: TYPE Root = T BRANDED Brand & " Root" OBJECT mu : MUTEX; (* as yet unused *) id : CARDINAL; tab : BDDTripleHash.T; cache : ARRAY Op OF BDDTripleHash.T; END; SchemeObject.T is the same as REFANY I'm not even sure where to start debugging this. The code "should work"... Indeed: > (rttype-typecode a) 144 > (RTBrand.GetByTC 144) "BDD 0.1 Root" > (RTBrand.GetByTC 143) "BDD 0.1" > (RTType.Supertype 143) 4 (4 is REFANY) > (BDD.And a a) EXCEPTION! RuntimeError! An explicit or implicit NARROW() operation failed. > a-and-a > (rttype-typecode a-and-a) 144 Mika From jay.krell at cornell.edu Sun Jan 19 02:02:56 2014 From: jay.krell at cornell.edu (Jay) Date: Sat, 18 Jan 2014 17:02:56 -0800 Subject: [M3devel] Win32 SuspendThread? In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL> References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL> Message-ID: <9EC3F463-F248-4C79-AF3F-130D2252B8B0@gmail.com> For native I'm less worried now. For wow64 I'm still worried & hope to follow up further. - Jay On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" wrote: > Jay: > > I?m very concerned about the threading not working properly on both 32-bit and 64-bit Windows. > > The Thread Test program crashes for me on both platforms. > > I haven?t tried your new test program yet. > > --Randy > > From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K > Sent: Saturday, January 18, 2014 1:23 AM > To: m3devel > Subject: EXT:[M3devel] Win32 SuspendThread? > > This program also doesn't behave as expected, native, nothing to do with wow64. > Anyone else please confirm: > 1) my expectations -- it should never print anything > 2) their importance -- garbage collector depends on it > 3) their not being met -- stuff gets printed > This is in the CVS repository, scratch/wow64stack/sync2.cpp > I am following up further. > Maybe we should get cooperative suspend really going? > Thank you. > - Jay > > #include > #include > volatile long value; > unsigned long __stdcall Thread(PVOID parameter) > { > while (1) > InterlockedIncrement(&value); > return 0; > } > int __cdecl main() > { > HANDLE thread = CreateThread(0, 0, Thread, 0, 0, 0); > UINT i = 0; > while (1) > { > i += 1; > if (SuspendThread(thread) == (DWORD)-1) > { > printf("suspend failed %X\n", GetLastError()); > Sleep(1); > continue; > } > volatile long a = value; > volatile long b = value; > if (a != b) > { > printf("%d %d %d %d\n", i, a, b, b - a); > } > ResumeThread(thread); > } > } -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Sun Jan 19 10:41:39 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sun, 19 Jan 2014 01:41:39 -0800 Subject: [M3devel] something wrong with ISTYPE ... Message-ID: <20140119094139.3917E1A208F@async.async.caltech.edu> Hi m3devel, There is definitely something wrong with ISTYPE. The problem I reported earlier goes away if I change the code like so: (416)truffles:~/t/btc/hparse/AMD64_LINUX>diff -c BDDSchemeStubs.m3.old BDDSchemeStubs.m3 *** BDDSchemeStubs.m3.old 2014-01-19 01:13:33.000000000 -0800 --- BDDSchemeStubs.m3 2014-01-19 01:36:52.000000000 -0800 *************** *** 613,622 **** PROCEDURE ToModula_BDD_T(x : SchemeObject.T) : (BDD.T) RAISES { Scheme.E } = BEGIN IF x # NIL AND ISTYPE(x, REF BDD.T) THEN RETURN NARROW(x,REF BDD.T)^ END; ! IF NOT ISTYPE(x,BDD.T) THEN RAISE Scheme.E("Not of type BDD.T : " & SchemeUtils.Stringify(x)) ! END; ! RETURN x END ToModula_BDD_T; --- 613,624 ---- PROCEDURE ToModula_BDD_T(x : SchemeObject.T) : (BDD.T) RAISES { Scheme.E } = BEGIN IF x # NIL AND ISTYPE(x, REF BDD.T) THEN RETURN NARROW(x,REF BDD.T)^ END; ! TYPECASE x OF ! BDD.T(xx) => RETURN xx ! ELSE RAISE Scheme.E("Not of type BDD.T : " & SchemeUtils.Stringify(x)) ! END ! END ToModula_BDD_T; This shouldn't be possible... it's the same object as before, with the same return type. I don't get an exception (I shouldn't either)---the return just behaves normally and the program works. Why doesn't it work with ISTYPE but only with TYPECASE? BTW, this is an old bug. Same behavior with ancient PM3. Where to look in the code? Mika From adacore at marino.st Sun Jan 19 11:03:27 2014 From: adacore at marino.st (John Marino) Date: Sun, 19 Jan 2014 11:03:27 +0100 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) Message-ID: <52DBA2EF.4020101@marino.st> Hi all, I've been trying to port cm3 to DragonFly x86-64. I'm able to build cm3cg on DragonFly, and use it to build libraries (e.g. libm3core) that were targeted for AMD64_FREEBSD. However, when I set the HOST to AMD64_FREEBSD and TARGET to AMD64_DRAGONFLY, I get the following error, "Fatal Error: unrecognized target machine: TARGET = AMD64_DRAGONFLY" (context below) ========================================================= > /work/lang/modula3/work/modula3-5.8.6/scripts/pkgmap.sh -c "/work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS " m3core > ROOT = /work/lang/modula3/work/modula3-5.8.6 > CM3_ROOT = /work/lang/modula3/work/modula3-5.8.6 > M3GDB = yes > M3OSTYPE = POSIX > TARGET = AMD64_DRAGONFLY > GCC_BACKEND = yes > INSTALLROOT = /work/lang/modula3/work/stage/usr/local > PKGSDB = /work/lang/modula3/work/modula3-5.8.6/scripts/PKGS > GREP = egrep > GMAKE = gmake > TMPDIR = /var/tmp > EXE = > SL = / > TAR = tar > CM3ROOT = /work/lang/modula3/work/modula3-5.8.6 > CM3VERSION = 5.8.6 > CM3VERSIONNUM = 050806 > CM3LASTCHANGED = 2010-04-11 > FIND = /usr/bin/find > EGREP = egrep > === package m3-libs/m3core === > +++ /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS +++ > --- building in AMD64_DRAGONFLY --- > > ignoring ../src/m3overrides > > > Fatal Error: unrecognized target machine: TARGET = AMD64_DRAGONFLY > > *** execution of /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS failed *** > *** [assembler-code] Error code 1 ========================================================= I thought I added all the target definitions but apparently I've missed something. I also realize the current repo looks a lot different, but I wanted to use the last official release. I've attached a concatenated file of patches showing what I've changed. Could somebody suggest what data is missing that causes the error above? Regards, John -------------- next part -------------- --- m3-libs/libm3/src/os/POSIX/m3makefile.orig 2009-07-24 10:02:40.000000000 +0000 +++ m3-libs/libm3/src/os/POSIX/m3makefile @@ -84,6 +84,7 @@ local readonly DefaultOSName = { "MIPS64_OPENBSD" : "OpenBSD", "SPARC64_SOLARIS" : "Solaris", "I386_OPENBSD" : "OpenBSD", + "AMD64_DRAGONFLY" : "DragonFly", "AMD64_FREEBSD" : "FreeBSD", "PA32_HPUX" : "HPUX", "PA64_HPUX" : "HPUX", @@ -92,6 +93,7 @@ local readonly DefaultOSName = { "AIX" : "AIX", "CYGWIN" : "Cygwin", "DARWIN" : "Darwin", + "DRAGONFLY" : "DragonFly", "FREEBSD" : "FreeBSD", "HPUX" : "HPUX", "INTERIX" : "Interix", @@ -156,6 +158,7 @@ local readonly DefaultArch = { "SPARC64_SOLARIS" : "sparc64", "I386_OPENBSD" : "i686", "AMD64_FREEBSD" : "AMD64", + "AMD64_DRAGONFLY" : "AMD64", "PA32_HPUX" : "hppa", "PA64_HPUX" : "hppa64", --- m3-libs/m3core/src/Csupport/m3makefile.orig 2009-07-24 10:51:01.000000000 +0000 +++ m3-libs/m3core/src/Csupport/m3makefile @@ -11,6 +11,7 @@ readonly _CSupportPieces = { % "ALPHA_OSF" : [ "little-endian" ], % "AIX386" : [ "little-endian" ], "AMD64_DARWIN": [ "little-endian" ], + "AMD64_DRAGONFLY": [ "little-endian" ], "AMD64_FREEBSD": [ "little-endian" ], "AMD64_OPENBSD": [ "little-endian" ], "AMD64_NETBSD": [ "little-endian" ], --- m3-libs/m3core/src/float/m3makefile.orig 2009-07-24 10:51:24.000000000 +0000 +++ m3-libs/m3core/src/float/m3makefile @@ -13,6 +13,7 @@ _FloatPieces = { % "ALPHA_OSF" : _float_le, % "AIX386" : _float_le, "AMD64_DARWIN": _float_le, + "AMD64_DRAGONFLY": _float_le, "AMD64_FREEBSD": _float_le, "AMD64_OPENBSD": _float_le, "AMD64_NETBSD": _float_le, --- m3-libs/m3core/src/m3core.h.orig 2010-03-28 10:07:45.000000000 +0000 +++ m3-libs/m3core/src/m3core.h @@ -207,7 +207,8 @@ typedef INTEGER m3_uid_t; #define PTHREAD_TO_M3(x) ((m3_pthread_t)(size_t)(x)) #define PTHREAD_FROM_M3(x) ((pthread_t)(size_t)(x)) -#if defined(__APPLE__) || defined(__FreeBSD__) || defined(__NetBSD__) || defined(__OpenBSD__) +#if defined(__APPLE__) || defined(__FreeBSD__) || defined(__NetBSD__) \ + || defined(__OpenBSD__) || defined(__DragonFly__) #define HAS_STAT_FLAGS #endif --- m3-libs/m3core/src/platforms.quake.orig 2010-03-28 10:09:07.000000000 +0000 +++ m3-libs/m3core/src/platforms.quake @@ -1,5 +1,6 @@ _TARGET_OS = { "AMD64_DARWIN" : "DARWIN", + "AMD64_DRAGONFLY" : "DRAGONFLY", "AMD64_FREEBSD" : "FREEBSD", "AMD64_LINUX" : "LINUX", "AMD64_NETBSD" : "NETBSD", @@ -32,6 +33,7 @@ _TARGET_ENDIAN = { % This is not computed based on architecture % because some architectures vary, e.g. PPC_NT is little endian, PPC_DARWIN is big. "AMD64_DARWIN" : "LITTLE", + "AMD64_DRAGONFLY" : "LITTLE", "AMD64_FREEBSD" : "LITTLE", "AMD64_LINUX" : "LITTLE", "AMD64_NETBSD" : "LITTLE", --- m3-libs/m3core/src/runtime/POSIX/RTSignalC.c.orig 2010-04-16 12:20:56.000000000 +0000 +++ m3-libs/m3core/src/runtime/POSIX/RTSignalC.c @@ -167,7 +167,7 @@ static size_t GetPC(void* xcontext) #elif defined(__NetBSD__) _UC_MACHINE_PC(context) -#elif defined(__FreeBSD__) +#elif defined(__FreeBSD__) || defined(__DragonFly__) #if defined(__amd64) context->uc_mcontext.mc_rip #elif defined(__i386) --- m3-libs/m3core/src/runtime/common/Compiler.tmpl.orig 2009-07-13 09:09:35.000000000 +0000 +++ m3-libs/m3core/src/runtime/common/Compiler.tmpl @@ -21,7 +21,7 @@ readonly Compiler_i3 = [ " SPARC64_OPENBSD, PPC32_OPENBSD, MIPS64_OPENBSD,", " SPARC64_SOLARIS, I386_OPENBSD, AMD64_FREEBSD,", " PA32_HPUX, PA64_HPUX, ARM_DARWIN, I386_INTERIX,", - " AMD64_NETBSD, AMD64_OPENBSD };", + " AMD64_NETBSD, AMD64_OPENBSD, AMD64_DRAGONFLY };", "", "CONST", " ThisOS = OS." & OS_TYPE & ";", --- m3-libs/m3core/src/thread.quake.orig 2009-12-20 11:01:02.000000000 +0000 +++ m3-libs/m3core/src/thread.quake @@ -12,6 +12,7 @@ readonly NO_USER_THREADS = "AMD64_FREEBSD":1, "AMD64_OPENBSD":1, "AMD64_NETBSD":1, + "AMD64_DRAGONFLY":1, "AMD64_LINUX":1, "ARM_LINUX":1, "MIPS32_IRIX":1, --- m3-libs/m3core/src/thread/PTHREAD/ThreadFreeBSD.c.orig 2009-12-21 05:35:34.000000000 +0000 +++ m3-libs/m3core/src/thread/PTHREAD/ThreadFreeBSD.c @@ -2,7 +2,7 @@ /* All rights reserved. */ /* See the file COPYRIGHT-PURDUE for a full description. */ -#ifndef __FreeBSD__ +#if !defined(__FreeBSD__) && !defined(__DragonFly__) /* avoid empty file */ --- m3-libs/m3core/src/thread/PTHREAD/ThreadPThreadC.c.orig 2009-12-12 14:48:10.000000000 +0000 +++ m3-libs/m3core/src/thread/PTHREAD/ThreadPThreadC.c @@ -23,7 +23,7 @@ #endif #endif -#ifdef __FreeBSD__ +#if defined(__FreeBSD__) || defined(__DragonFly__) #define M3_DIRECT_SUSPEND #endif @@ -177,7 +177,7 @@ void ThreadPThread__sem_post(void) void ThreadPThread__sem_getvalue(void) { M3_DIRECT_SUSPEND_ASSERT_FALSE } void ThreadPThread__sigsuspend(void) { M3_DIRECT_SUSPEND_ASSERT_FALSE } -#ifdef __FreeBSD__ +#if defined(__FreeBSD__) || defined(__DragonFly__) int ThreadPThread__SuspendThread (m3_pthread_t mt) --- m3-libs/m3core/src/time/POSIX/m3makefile.orig 2009-07-24 07:26:15.000000000 +0000 +++ m3-libs/m3core/src/time/POSIX/m3makefile @@ -11,6 +11,7 @@ readonly _DateImpls = { % "ALPHA_OSF" : "DateBsd", % "AIX386" : "DateBsd", "AMD64_DARWIN": "DateBsd", + "AMD64_DRAGONFLY": "DateBsd", "AMD64_FREEBSD": "DateBsd", "AMD64_OPENBSD": "DateBsd", "AMD64_NETBSD": "DateBsd", @@ -63,6 +64,7 @@ readonly _DateImpls = { % "AIX" : "DateBsd", "CYGWIN" : "DatePosix", "DARWIN" : "DateBsd", + "DRAGONFLY" : "DataBsd", "FREEBSD" : "DateBsd", "HPUX" : "DatePosix", "INTERIX" : "DatePosix", --- m3-libs/m3core/src/unix/Common/UtimeC.c.orig 2009-12-11 11:18:34.000000000 +0000 +++ m3-libs/m3core/src/unix/Common/UtimeC.c @@ -56,7 +56,8 @@ struct timeval #endif } -#if defined(__OpenBSD__) || defined(__FreeBSD__) || defined(__NetBSD__) || defined(__APPLE__) +#if defined(__OpenBSD__) || defined(__FreeBSD__) || defined(__NetBSD__) \ + || defined(__APPLE__) || defined(__DragonFly__) #define M3BSD #endif --- m3-libs/m3core/src/unix/Common/Uucontext.c.orig 2009-06-29 19:20:40.000000000 +0000 +++ m3-libs/m3core/src/unix/Common/Uucontext.c @@ -29,6 +29,7 @@ see http://www.opengroup.org/onlinepubs/ || defined(__linux) \ || defined(__NetBSD__) \ || defined(__hpux) \ + || defined(__DragonFly__) \ || defined(__FreeBSD__) \ int Uucontext__getcontext(ucontext_t* context) --- m3-libs/m3core/src/unix/Common/m3makefile.orig 2009-07-19 00:28:17.000000000 +0000 +++ m3-libs/m3core/src/unix/Common/m3makefile @@ -81,7 +81,7 @@ if equal (TARGET, "MIPS64_OPENBSD") or e or equal (TARGET, "PA32_HPUX") or equal (TARGET, "PA64_HPUX") or equal (TARGET, "NT386") or equal (TARGET, "AMD64_FREEBSD") or equal (TARGET, "AMD64_OPENBSD") or equal (TARGET, "AMD64_NETBSD") - or equal (TARGET, "I386_INTERIX") + or equal (TARGET, "I386_INTERIX") or equal (TARGET, "AMD64_DRAGONFLY") Interface("Uucontext") --- m3-libs/m3core/src/unix/m3makefile.orig 2009-07-28 18:49:13.000000000 +0000 +++ m3-libs/m3core/src/unix/m3makefile @@ -6,6 +6,7 @@ readonly _UnixPieces = { % "AIX386" : [ "aix-ps2-1-2", "uin-common" ], % "ALPHA_OSF" : [ "osf-1.generic", "osf-1.ALPHA_OSF", "uin-common" ], "AMD64_DARWIN" : [ "darwin-common", "darwin-amd64", "uin-len" ], + "AMD64_DRAGONFLY" : [ "freebsd-common", "uin-len" ], "AMD64_FREEBSD" : [ "freebsd-common", "uin-len" ], "AMD64_LINUX" : [ "linux-common", "uin-common" ], "AMD64_NETBSD" : [ "netbsd-common", "uin-len" ], --- m3-sys/cminstall/src/config-no-install/AMD64_DRAGONFLY.orig 2014-01-16 23:14:23.000000000 +0000 +++ m3-sys/cminstall/src/config-no-install/AMD64_DRAGONFLY @@ -0,0 +1,9 @@ +readonly TARGET = "AMD64_DRAGONFLY" +readonly GNU_PLATFORM = "amd64-dragonfly" % "cpu-os" string for GNU + +m3back_flags = "-gstabs+ -m64 -fPIC -funwind-tables" +SYSTEM_CC = "gcc -gstabs+ -m64 -fPIC" +readonly SYSTEM_ASM = "as -64" + +include("AMD64.common") +include("DragonFly.common") --- m3-sys/cminstall/src/config-no-install/DragonFly.common.orig 2014-01-16 23:18:36.000000000 +0000 +++ m3-sys/cminstall/src/config-no-install/DragonFly.common @@ -0,0 +1,24 @@ +readonly TARGET_OS = "DRAGONFLY" + +GNU_MAKE = "gmake" + +include("Unix.common") + +SYSTEM_LIBS{"ODBC"} = [ "-Wl,-R/usr/local/lib", "-L/usr/local/lib", "-lodbc" ] +SYSTEM_LIBS{"POSTGRES95"} = [ "-Wl,-R/usr/local/lib", "-L/usr/local/lib", "-lpq" ] +SYSTEM_LIBS{"X11"} = ["-Wl,-R/usr/local/lib", "-L/usr/local/lib", "-lXaw", "-lXmu", "-lXext", "-lXt", "-lSM", "-lICE", "-lX11" ] + +DRAGONFLY_CC_APPEND = " -z origin" + & " -z now" + & " -Bsymbolic" + +DRAGONFLY_LD_APPEND = DRAGONFLY_CC_APPEND + & " -Wl,--fatal-warnings" + & " -Wl,--warn-common" + & " -Wl,-rpath,/usr/local/lib" + & " -Wl,-rpath,\\$ORIGIN/../lib " + +SYSTEM_LD = SYSTEM_CC & DRAGONFLY_LD_APPEND +SYSTEM_CC = SYSTEM_CC & DRAGONFLY_CC_APPEND + +include("gnuld.common") --- m3-sys/m3cc/src/boot.py.orig 2008-05-19 13:09:03.000000000 +0000 +++ m3-sys/m3cc/src/boot.py @@ -76,7 +76,7 @@ if SearchPath("gcc"): " + env) Run(gmake + "all-gmp all-mpfr all-libcpp all-libdecnumber \ - all-build-libiberty all-libiberty configure-gcc") + all-build-libiberty all-libiberty configure-libcpp configure-gcc") os.chdir("gcc") --- m3-sys/m3cc/src/m3makefile.orig 2010-05-14 07:00:52.000000000 +0000 +++ m3-sys/m3cc/src/m3makefile @@ -577,7 +577,7 @@ end if GCC42 m3cc_Run (["cd " & build_dir & " && " & M3CC_MAKE & " all-build-libiberty all-libiberty all-libdecnumber | tee -a " & Log]) else - m3cc_Run (["cd " & build_dir & " && " & M3CC_MAKE & " all-gmp all-mpfr all-build-libiberty all-libiberty all-libdecnumber | tee -a " & Log]) + m3cc_Run (["cd " & build_dir & " && " & M3CC_MAKE & " all-gmp all-mpfr all-build-libiberty all-libiberty all-libdecnumber configure-libcpp | tee -a " & Log]) end % --- m3-sys/m3cc/src/platforms.quake.orig 2009-07-29 17:11:08.000000000 +0000 +++ m3-sys/m3cc/src/platforms.quake @@ -13,6 +13,7 @@ readonly Platform_info = { "ALPHA_OSF" : "alpha-dec-osf1", "ALPHA_VMS" : "alpha-vms", "AMD64_DARWIN" : "i686-apple-darwin8", + "AMD64_DRAGONFLY" : "amd64-dragonfly", "AMD64_FREEBSD" : "amd64-freebsd7", "AMD64_LINUX" : "amd64-pc-linux-gnu", "AMD64_NETBSD" : "amd64-netbsd", --- m3-sys/m3gdb/src/platforms.quake.orig 2010-04-21 17:06:38.000000000 +0000 +++ m3-sys/m3gdb/src/platforms.quake @@ -13,6 +13,7 @@ readonly Platform_info = { "ALPHA_OSF" : "alpha-dec-osf1", "ALPHA_VMS" : "alpha-vms", "AMD64_DARWIN" : "amd64-apple-darwin8.7.1", + "AMD64_DRAGONFLY" : "amd64-dragonfly", "AMD64_FREEBSD" : "amd64-freebsd", "AMD64_LINUX" : "amd64-pc-linux-gnu", "AMD64_NETBSD" : "amd64-netbsd", --- m3-sys/m3middle/src/Target.i3.orig 2010-02-01 14:05:35.000000000 +0000 +++ m3-sys/m3middle/src/Target.i3 @@ -33,7 +33,7 @@ TYPE AMD64_DARWIN, AMD64_LINUX, SPARC32_LINUX, SPARC64_LINUX, SPARC64_OPENBSD, PPC32_OPENBSD, MIPS64_OPENBSD, SPARC64_SOLARIS, I386_OPENBSD, AMD64_FREEBSD, PA32_HPUX, PA64_HPUX, ARM_DARWIN, - I386_INTERIX, AMD64_NETBSD, AMD64_OPENBSD, Undefined + I386_INTERIX, AMD64_NETBSD, AMD64_OPENBSD, AMD64_DRAGONFLY, Undefined }; CONST @@ -89,7 +89,8 @@ CONST (* 48 *) "ARM_DARWIN", (* 49 *) "I386_INTERIX", (* 50 *) "AMD64_NETBSD", - (* 51 *) "AMD64_OPENBSD" + (* 51 *) "AMD64_OPENBSD", + (* 52 *) "AMD64_DRAGONFLY" }; CONST --- m3-sys/m3middle/src/Target.m3.orig 2010-02-01 15:06:05.000000000 +0000 +++ m3-sys/m3middle/src/Target.m3 @@ -307,6 +307,7 @@ PROCEDURE Init (system: TEXT; in_OS_name | Systems.AMD64_NETBSD, Systems.AMD64_OPENBSD, + Systems.AMD64_DRAGONFLY, Systems.AMD64_FREEBSD => Jumpbuf_size := 16_60 * Char.size; --- m3-tools/cvsup/suplib/src/FreeBSD/UProcTitle.c.orig 2009-04-12 10:31:23.000000000 +0000 +++ m3-tools/cvsup/suplib/src/FreeBSD/UProcTitle.c @@ -1,11 +1,11 @@ -#ifdef __FreeBSD__ +#if defined __FreeBSD__) || defined(__DragonFly__) #include #include #endif void m3_setproctitle(const char *title) { -#ifdef __FreeBSD__ +#if defined(__FreeBSD__) || defined(__DragonFly__) setproctitle("%s", title); #endif } --- scripts/boot-cm3-makefile-lib.tmpl.orig 2005-04-07 22:35:07.000000000 +0000 +++ scripts/boot-cm3-makefile-lib.tmpl @@ -20,9 +20,10 @@ FILES_C := $(wildcard *.c) FILES_S := $(wildcard *.s) BASES_IC := $(basename $(FILES_IC)) BASES_MC := $(basename $(FILES_MC)) +BASES_MS := $(basename $(FILES_MS)) BASES_C := $(basename $(FILES_C) $(FILES_S)) OBJ_IO := $(addsuffix .io, $(BASES_IC)) -OBJ_MO := $(addsuffix .mo, $(BASES_MC)) +OBJ_MO := $(addsuffix .mo, $(BASES_MS)) OBJ_O := $(addsuffix .o, $(BASES_C)) OBJECTS := $(OBJ_IO) $(OBJ_MO) $(OBJ_O) --- scripts/boot-cm3-makefile-prog.tmpl.orig 2003-07-10 22:16:05.000000000 +0000 +++ scripts/boot-cm3-makefile-prog.tmpl @@ -19,9 +19,10 @@ FILES_MS := $(wildcard *.ms) FILES_C := $(wildcard *.c) BASES_IC := $(basename $(FILES_IC)) BASES_MC := $(basename $(FILES_MC)) +BASES_MS := $(basename $(FILES_MS)) BASES_C := $(basename $(FILES_C)) OBJ_IO := $(addsuffix .io, $(BASES_IC)) -OBJ_MO := $(addsuffix .mo, $(BASES_MC)) +OBJ_MO := $(addsuffix .mo, $(BASES_MS)) OBJ_O := $(addsuffix .o, $(BASES_C)) OBJECTS := $(OBJ_IO) $(OBJ_MO) $(OBJ_O) LIBFILES := $(shell for f in $(LIBS); do if [ -f "$${f}" ] ; then echo $${f}; fi; done) --- scripts/pack-crossbuild.sh.orig 2010-02-25 14:02:18.000000000 +0000 +++ scripts/pack-crossbuild.sh @@ -23,7 +23,7 @@ if [ -z "$1" ] ; then fi CROSS_TARGET=$1 BUILDARGS='' -M3CONFIG=${ROOT}/m3-sys/cm3/src/config/${CROSS_TARGET} +M3CONFIG=${ROOT}/m3-sys/cminstall/src/config-no-install/${CROSS_TARGET} export M3CONFIG . "$ROOT/scripts/pkginfo.sh" . "$ROOT/scripts/pkgcmds.sh" @@ -38,25 +38,7 @@ P="${P} m3back" P="${P} m3front" P="${P} m3quake" P="${P} cm3" -P="${P} m3scanner" -P="${P} m3tools" -P="${P} m3cgcat" -P="${P} m3cggen" -[ "${M3GDB}" = yes ] && P="${P} m3gdb" P="${P} m3bundle" -P="${P} mklib" -P="${P} fix_nl" -P="${P} libdump" -P="${P} bitvector" -P="${P} digraph" -P="${P} parseparams" -P="${P} realgeometry" -P="${P} set" -P="${P} slisp" -P="${P} sortedtableextras" -P="${P} table-list" -P="${P} tempfiles" -[ "${HAVE_TCL}" = "yes" ] && P="${P} tcl" USAGE=" `basename $0` cross_target From jay.krell at cornell.edu Sun Jan 19 12:35:54 2014 From: jay.krell at cornell.edu (Jay K) Date: Sun, 19 Jan 2014 11:35:54 +0000 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: <52DBA2EF.4020101@marino.st> References: <52DBA2EF.4020101@marino.st> Message-ID: C:\dev2\cm3.2>findstr /s /i /p /c:"unrecognized target machine" *m3 m3-sys\cm3\src\Builder.m3: Msg.FatalError (NIL, "unrecognized target machin e: TARGET = ", s.target); Please don't bother with the last release. Please use the current CVS version. And please use the C backend. Did you edit m3-sys/m3middle/src/Target.i3 and possibly Target.m3? Please use this as a guide for files to consider changing: C:\dev2\cm3.2>findstr /m /s /i /p freebsd m3makefile m3-comm\udp\test\src\m3makefile m3-libs\libm3\src\os\POSIX\m3makefile m3-libs\m3core\src\float\m3makefile m3-libs\m3core\src\thread\m3makefile m3-libs\m3core\src\thread\PTHREAD\m3makefile m3-libs\m3core\src\unix\m3makefile m3-sys\m3cc\src\m3makefile m3-sys\m3gdb\src\m3makefile m3-tools\cvsup\suplib\src\FreeBSD\m3makefile m3-tools\cvsup\suplib\src\m3makefile m3-tools\cvsup\suptcp\src\POSIX\m3makefile C:\dev2\cm3.2>findstr /m /s /i /p freebsd *.c m3-libs\libm3\src\uid\POSIX\getMID.c m3-libs\libm3\src\uid\POSIX\MachineIDPosixC.c m3-libs\m3core\src\Csupport\Common\s_lroundl.c m3-libs\m3core\src\float\C99\FloatModeC.c m3-libs\m3core\src\runtime\common\RTProcessC.c m3-libs\m3core\src\runtime\POSIX\RTSignalC.c m3-libs\m3core\src\thread\POSIX\ThreadPosixC.c m3-libs\m3core\src\thread\PTHREAD\ThreadFreeBSD.c m3-libs\m3core\src\thread\PTHREAD\ThreadOpenBSD.c m3-libs\m3core\src\thread\PTHREAD\ThreadPThreadC.c m3-libs\m3core\src\unix\Common\Uin.c m3-libs\patternmatching\src\libglob\fnmatch.c esp. m3core and libm3. If we are sure of the name "AMD64_DRAGONFLY" I can go ahead and do it for you. But I realize there is much value in me NOT doing it. I thought maybe AMD64_DBSD or DFLYBSD, but DRAGFLY is ok. - Jay > Date: Sun, 19 Jan 2014 11:03:27 +0100 > From: adacore at marino.st > To: m3devel at elegosoft.com > Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) > > Hi all, > > I've been trying to port cm3 to DragonFly x86-64. > I'm able to build cm3cg on DragonFly, and use it to build libraries > (e.g. libm3core) that were targeted for AMD64_FREEBSD. > > However, when I set the HOST to AMD64_FREEBSD and TARGET to > AMD64_DRAGONFLY, I get the following error, "Fatal Error: unrecognized > target machine: TARGET = AMD64_DRAGONFLY" (context below) > > ========================================================= > > /work/lang/modula3/work/modula3-5.8.6/scripts/pkgmap.sh -c "/work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS " m3core > > ROOT = /work/lang/modula3/work/modula3-5.8.6 > > CM3_ROOT = /work/lang/modula3/work/modula3-5.8.6 > > M3GDB = yes > > M3OSTYPE = POSIX > > TARGET = AMD64_DRAGONFLY > > GCC_BACKEND = yes > > INSTALLROOT = /work/lang/modula3/work/stage/usr/local > > PKGSDB = /work/lang/modula3/work/modula3-5.8.6/scripts/PKGS > > GREP = egrep > > GMAKE = gmake > > TMPDIR = /var/tmp > > EXE = > > SL = / > > TAR = tar > > CM3ROOT = /work/lang/modula3/work/modula3-5.8.6 > > CM3VERSION = 5.8.6 > > CM3VERSIONNUM = 050806 > > CM3LASTCHANGED = 2010-04-11 > > FIND = /usr/bin/find > > EGREP = egrep > > === package m3-libs/m3core === > > +++ /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS +++ > > --- building in AMD64_DRAGONFLY --- > > > > ignoring ../src/m3overrides > > > > > > Fatal Error: unrecognized target machine: TARGET = AMD64_DRAGONFLY > > > > *** execution of /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS failed *** > > *** [assembler-code] Error code 1 > > ========================================================= > > I thought I added all the target definitions but apparently I've missed > something. I also realize the current repo looks a lot different, but I > wanted to use the last official release. > > I've attached a concatenated file of patches showing what I've changed. > Could somebody suggest what data is missing that causes the error above? > > Regards, > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Jan 19 12:40:17 2014 From: jay.krell at cornell.edu (Jay K) Date: Sun, 19 Jan 2014 11:40:17 +0000 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: References: <52DBA2EF.4020101@marino.st>, Message-ID: Your change looks about right. Did you first "upgrade" and existing install, with these changes, and then cross? That is what you must do. - Jay From: jay.krell at cornell.edu To: adacore at marino.st; m3devel at elegosoft.com Subject: RE: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) Date: Sun, 19 Jan 2014 11:35:54 +0000 C:\dev2\cm3.2>findstr /s /i /p /c:"unrecognized target machine" *m3 m3-sys\cm3\src\Builder.m3: Msg.FatalError (NIL, "unrecognized target machin e: TARGET = ", s.target); Please don't bother with the last release. Please use the current CVS version. And please use the C backend. Did you edit m3-sys/m3middle/src/Target.i3 and possibly Target.m3? Please use this as a guide for files to consider changing: C:\dev2\cm3.2>findstr /m /s /i /p freebsd m3makefile m3-comm\udp\test\src\m3makefile m3-libs\libm3\src\os\POSIX\m3makefile m3-libs\m3core\src\float\m3makefile m3-libs\m3core\src\thread\m3makefile m3-libs\m3core\src\thread\PTHREAD\m3makefile m3-libs\m3core\src\unix\m3makefile m3-sys\m3cc\src\m3makefile m3-sys\m3gdb\src\m3makefile m3-tools\cvsup\suplib\src\FreeBSD\m3makefile m3-tools\cvsup\suplib\src\m3makefile m3-tools\cvsup\suptcp\src\POSIX\m3makefile C:\dev2\cm3.2>findstr /m /s /i /p freebsd *.c m3-libs\libm3\src\uid\POSIX\getMID.c m3-libs\libm3\src\uid\POSIX\MachineIDPosixC.c m3-libs\m3core\src\Csupport\Common\s_lroundl.c m3-libs\m3core\src\float\C99\FloatModeC.c m3-libs\m3core\src\runtime\common\RTProcessC.c m3-libs\m3core\src\runtime\POSIX\RTSignalC.c m3-libs\m3core\src\thread\POSIX\ThreadPosixC.c m3-libs\m3core\src\thread\PTHREAD\ThreadFreeBSD.c m3-libs\m3core\src\thread\PTHREAD\ThreadOpenBSD.c m3-libs\m3core\src\thread\PTHREAD\ThreadPThreadC.c m3-libs\m3core\src\unix\Common\Uin.c m3-libs\patternmatching\src\libglob\fnmatch.c esp. m3core and libm3. If we are sure of the name "AMD64_DRAGONFLY" I can go ahead and do it for you. But I realize there is much value in me NOT doing it. I thought maybe AMD64_DBSD or DFLYBSD, but DRAGFLY is ok. - Jay > Date: Sun, 19 Jan 2014 11:03:27 +0100 > From: adacore at marino.st > To: m3devel at elegosoft.com > Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) > > Hi all, > > I've been trying to port cm3 to DragonFly x86-64. > I'm able to build cm3cg on DragonFly, and use it to build libraries > (e.g. libm3core) that were targeted for AMD64_FREEBSD. > > However, when I set the HOST to AMD64_FREEBSD and TARGET to > AMD64_DRAGONFLY, I get the following error, "Fatal Error: unrecognized > target machine: TARGET = AMD64_DRAGONFLY" (context below) > > ========================================================= > > /work/lang/modula3/work/modula3-5.8.6/scripts/pkgmap.sh -c "/work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS " m3core > > ROOT = /work/lang/modula3/work/modula3-5.8.6 > > CM3_ROOT = /work/lang/modula3/work/modula3-5.8.6 > > M3GDB = yes > > M3OSTYPE = POSIX > > TARGET = AMD64_DRAGONFLY > > GCC_BACKEND = yes > > INSTALLROOT = /work/lang/modula3/work/stage/usr/local > > PKGSDB = /work/lang/modula3/work/modula3-5.8.6/scripts/PKGS > > GREP = egrep > > GMAKE = gmake > > TMPDIR = /var/tmp > > EXE = > > SL = / > > TAR = tar > > CM3ROOT = /work/lang/modula3/work/modula3-5.8.6 > > CM3VERSION = 5.8.6 > > CM3VERSIONNUM = 050806 > > CM3LASTCHANGED = 2010-04-11 > > FIND = /usr/bin/find > > EGREP = egrep > > === package m3-libs/m3core === > > +++ /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS +++ > > --- building in AMD64_DRAGONFLY --- > > > > ignoring ../src/m3overrides > > > > > > Fatal Error: unrecognized target machine: TARGET = AMD64_DRAGONFLY > > > > *** execution of /work/lang/modula3/work/stage/usr/local/bin/cm3 -build -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3_VERSION_TEXT='5.8.6' -DCM3_VERSION_NUMBER='050806' -DCM3_LAST_CHANGED='2010-04-11' -DROOT='/work/lang/modula3/work/modula3-5.8.6' -DCM3VERSION='5.8.6' -DCM3VERSIONNUM='050806' -DCM3LASTCHANGED='2010-04-11' $RARGS failed *** > > *** [assembler-code] Error code 1 > > ========================================================= > > I thought I added all the target definitions but apparently I've missed > something. I also realize the current repo looks a lot different, but I > wanted to use the last official release. > > I've attached a concatenated file of patches showing what I've changed. > Could somebody suggest what data is missing that causes the error above? > > Regards, > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Sun Jan 19 12:53:53 2014 From: adacore at marino.st (John Marino) Date: Sun, 19 Jan 2014 12:53:53 +0100 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: References: <52DBA2EF.4020101@marino.st> Message-ID: <52DBBCD1.7020904@marino.st> On 1/19/2014 12:35, Jay K wrote: > C:\dev2\cm3.2>findstr /s /i /p /c:"unrecognized target machine" *m3 > m3-sys\cm3\src\Builder.m3: Msg.FatalError (NIL, "unrecognized > target machin > e: TARGET = ", s.target); > > > Please don't bother with the last release. > Please use the current CVS version. But the last release is what is in FreeBSD ports, the basis of both FreeBSD and DragonFly. And ports doesn't build from repos normally, although there is a facility to pull directly from github, but I don't like that method myself. If there was an official downloadable snapshot or release candidate, I could use that. Since 2010 was the last release, is another one coming soon? > And please use the C backend. > Did you edit m3-sys/m3middle/src/Target.i3 and possibly Target.m3? Yes, that was one of the many patches in the attachment I send in the previous mail. > Please use this as a guide for files to consider changing: > C:\dev2\cm3.2>findstr /m /s /i /p freebsd m3makefile > m3-comm\udp\test\src\m3makefile > m3-libs\libm3\src\os\POSIX\m3makefile > m3-libs\m3core\src\float\m3makefile > m3-libs\m3core\src\thread\m3makefile > m3-libs\m3core\src\thread\PTHREAD\m3makefile > m3-libs\m3core\src\unix\m3makefile > m3-sys\m3cc\src\m3makefile > m3-sys\m3gdb\src\m3makefile > m3-tools\cvsup\suplib\src\FreeBSD\m3makefile > m3-tools\cvsup\suplib\src\m3makefile > m3-tools\cvsup\suptcp\src\POSIX\m3makefile The only ones I haven't patched already are m3-libs\m3core\src\thread\PTHREAD\m3makefile and the cvsup ones. Those wouldn't cause the error. I also assume this is the top of the repo. > C:\dev2\cm3.2>findstr /m /s /i /p freebsd *.c > m3-libs\libm3\src\uid\POSIX\getMID.c > m3-libs\libm3\src\uid\POSIX\MachineIDPosixC.c > m3-libs\m3core\src\Csupport\Common\s_lroundl.c > m3-libs\m3core\src\float\C99\FloatModeC.c > m3-libs\m3core\src\runtime\common\RTProcessC.c > m3-libs\m3core\src\runtime\POSIX\RTSignalC.c > m3-libs\m3core\src\thread\POSIX\ThreadPosixC.c > m3-libs\m3core\src\thread\PTHREAD\ThreadFreeBSD.c > m3-libs\m3core\src\thread\PTHREAD\ThreadOpenBSD.c > m3-libs\m3core\src\thread\PTHREAD\ThreadPThreadC.c > m3-libs\m3core\src\unix\Common\Uin.c > m3-libs\patternmatching\src\libglob\fnmatch.c > > > esp. m3core and libm3. I haven't seen freebsd in these, so must be a difference between release and repo. > If we are sure of the name "AMD64_DRAGONFLY" I can go ahead and do it > for you. > But I realize there is much value in me NOT doing it. > I thought maybe AMD64_DBSD or DFLYBSD, but DRAGFLY is ok. Because AMD64_DRAGONFLY is too long? I'd prefer to keep it "AMD64_DRAGONFLY" because it matches the other targets and because of all the work I've already done. I'm really, really close to getting cm3 bootstrapped on DragonFly with 5.8.6. I believe if this "M3_BOOTSTRAP = true" version would compile, I could build cm3 and libraries on DragonFly with the cm3cg that's already built on it. Once I have a fully native compiler, then I can update to the latest repo, especially if some kind of snapshot is released. Thanks, John From jay.krell at cornell.edu Sun Jan 19 13:00:01 2014 From: jay.krell at cornell.edu (Jay K) Date: Sun, 19 Jan 2014 12:00:01 +0000 Subject: [M3devel] Cooperative suspend? In-Reply-To: <8BC1682A-3760-44D0-BA67-3A0F3EA10E27@m3w.org> References: <8BC1682A-3760-44D0-BA67-3A0F3EA10E27@m3w.org> Message-ID: Windows uses SuspendThread. I'll double check. Cooperative suspend would NOT involve sending a "signal" to threads, in the Posix sense. Nor would it use SuspendThread. The model for cooperative suspend has been described by Tony and goes something like this: There is a list of threads maintained by the m3 runtime. Not by enumerating OS threads. The thread structure contains in it perhaps a boolean or enum as to suspend requested, suspending, suspended, resuming, running, etc. When a thread wants to collect garbage, it either walks the thread list and sets all the booleans/enums to "suspend requested", or, more likely, it sets one global boolean. Now, there are three likely methods to set the global boolean. 1 - it is a global writable variable 2 - it is a global writable function pointer 3 - it is a page with protection Code frequently polls this global boolean, in one of the following fashions 1 - read and test the value of the variable 2 - call the function pointer 3 - read the page (just a byte or a word) ignoring the contents 4 - call a function to do any of the above, esp #1 #3 is probably fastest but my least favorite. It has lingering portability problems and is slowest when it comes time to actually suspend -- catch the exception and for purposes of debugging, you really don't want a design that raises exceptions like that. When does the polling occur? The polling is generated by the compiler. It is done at function entry and/or exit and at branches or at least backward branches or at least branches that are continuing loops or such or at the starts of loops or somesuch. I believe Java uses cooperative suspend and I think it uses the page protection mechanism. It isn't portable e.g. to systems without an MMU. Though of course you could abstract it out. I'm glossing over details, but you know all the code we have for signals and semaphores? It goes away. Our portability then would drive one more significant notch toward that of "hello world". In my mine, we have about two pieces to do to gain "hello world" level of portability - cooperative suspend, not easy - jmpbuf size removal, should be easy and some small matters - endianness, particularly in the libary - word size - Jay From: dragisha at m3w.org Date: Sat, 18 Jan 2014 09:24:11 +0100 To: m3devel at elegosoft.com Subject: [M3devel] Cooperative suspend? Jay, What exactly do you mean? Isn't it cooperative already? Thread stopping the world informs others about its intent, and they suspend themselves on a semaphore. Windows implementation of threading does forced suspend? dd --Dragi?a Duri?dragisha at m3w.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Sun Jan 19 13:09:37 2014 From: adacore at marino.st (John Marino) Date: Sun, 19 Jan 2014 13:09:37 +0100 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: References: <52DBA2EF.4020101@marino.st>, Message-ID: <52DBC081.1070401@marino.st> On 1/19/2014 12:40, Jay K wrote: > Your change looks about right. > > Did you first "upgrade" and existing install, with these changes, and > then cross? > That is what you must do. It's probably easier that I show you. I've attached "Makefile.local" (I added the .txt extension just now to help out some mail clients) which is my recipe. So I run three targets in a freshly patched workarea: 1) make cross-compiler 2) make assembler-code (called from cross-archive) 3) make cross-archive In the cross-compiler target, the FreeBSD bootstrap compiler builds m3middle, m3linker, m3front, mequake, and CM3 for the cross compiler. The cross-compiler then builds libm3core and libm3 with FreeBSD cm3cg Everything is cleaned, then the cross compiler builds everything again (including libm3core and libm3) along with m3objfile, sysutils, and patternmatching. Then the cm3.cfg is changed to specify HOST=freebsd, TARGET=dragonfly, cm3cg is cm3cg-AMD64_DRAGONFLY. I would not be surprised is there is a logic flaw or two in "make cross-compiler". The "how to port CM3" tutorial was slightly out of date and some of the instructions were a bit ambiguous so this was a best guess. Regards, John -------------- next part -------------- # This makefile fragment is automatically included by bsd.port.mk # It is separate because it contains maintenance targets not used in # normal package building, but the recipes are too value to lose. So # we keep them with the port, but out of the way of regular users. # # $FreeBSD$ ## # # The "create-bootstrap" target will assemble a new bootstrap based on the # contents of the stage directory, which means it has to be run after the # "stage" target. # ## NEWBOOTDIR= ${WRKDIR}/new-bootstrap/bootstrap NEWBOOTNAME= m3-bootstrap.${MARCH}.${OPSYS:U}.${OSREL:S/.//}.tar.bz2 BSCONTENTS= bin/cm3 bin/cm3cg bin/m3bundle bin/mklib etc/modula3 \ lib/libm3core.* lib/libm3.* lib/libsysutils.* \ lib/libpatternmatching.* pkg/m3core pkg/libm3 pkg/sysutils \ pkg/patternmatching pkg/m3middle pkg/m3objfile pkg/m3linker \ pkg/m3back pkg/m3front pkg/m3quake pkg/cm3 pkg/mklib create-bootstrap: @${RM} -rf ${NEWBOOTDIR} @${MKDIR} ${NEWBOOTDIR}/bin ${NEWBOOTDIR}/lib \ ${NEWBOOTDIR}/pkg ${NEWBOOTDIR}/etc .for X in ${BSCONTENTS} @${CP} -a ${STAGEDIR}${PREFIX}/${X} ${NEWBOOTDIR}/${X:H}/ .endfor ${ECHO} "INSTALL_ROOT = path() & \"/..\"" \ > ${NEWBOOTDIR}/bin/cm3.cfg ${ECHO} "include(path() & \"/../etc/modula3/${M3TARGET}\")" \ >> ${NEWBOOTDIR}/bin/cm3.cfg @${FIND} ${NEWBOOTDIR} -type f -perm +111 | ${XARGS} ${STRIP_CMD} (cd ${NEWBOOTDIR}/.. ; tar -cyf ${NEWBOOTNAME} bootstrap) ## # # The "cross-compiler" target will create a cross-compiler in the # stage directory based on the value of ${CROSSTARG}. # Execute it after the "patch" target # # The "cross-archive" will archive assembler code meant for linking # on the native plaform. It is run after "cross-compiler" # # The "native-link" target will bootstrap cm3 on a new platform (which # obviously supports ports) using the archived files generated above. # It could be done remotely via SSH, but this one target assumes files # are locally available at ${WRKSRC}/${CROSSTARG} # ## CROSSTARG?= AMD64_DRAGONFLY CORELIBS= m3-libs/m3core m3-libs/libm3 CORESYS= m3-sys/m3middle m3-sys/m3objfile m3-sys/m3linker \ m3-sys/m3back m3-sys/m3front m3-sys/m3quake m3-sys/cm3 \ m3-tools/m3bundle COREHELPERS= m3-libs/sysutils m3-libs/patternmatching COREFILES= ${CORELIBS} ${CORESYS} cross-compiler: @(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BOOTCM3} SHIP=${TRUE} \ ${SH} scripts/build-cross-backend.sh ${CROSSTARG}) @(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BOOTCM3} SHIP=${TRUE} \ ${SH} scripts/do-pkg.sh onlybuild \ m3middle m3linker m3front m3quake cm3) @${FIND} ${WRKSRC} -name \.M3SHIP -print | ${XARGS} ${SED} -i -e \ 's|/bootstrap/|/stage${PREFIX}/|g' @(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BOOTCM3} \ ${SH} scripts/do-pkg.sh ship \ m3middle m3linker m3front m3quake cm3) (cd ${WRKSRC}/m3-sys/cminstall/src/config-no-install && \ ${COPYTREE_SHARE} . ${STAGEDIR}${PREFIX}/etc/modula3) ${MKDIR} ${STAGEDIR}${PREFIX}/bin ${INSTALL_PROGRAM} ${WRKSRC}/m3-sys/cm3/${M3TARGET}/cm3 \ ${STAGEDIR}${PREFIX}/bin ${ECHO} "INSTALL_ROOT = \"${STAGEDIR}${PREFIX}\"" > \ ${STAGEDIR}${PREFIX}/bin/cm3.cfg ${ECHO} "include(\"${STAGEDIR}${PREFIX}/etc/modula3/${M3TARGET}\")" >> \ ${STAGEDIR}${PREFIX}/bin/cm3.cfg ${INSTALL_PROGRAM} ${WRKDIR}/bootstrap/bin/cm3cg \ ${STAGEDIR}${PREFIX}/bin/ ${INSTALL_PROGRAM} \ ${WRKSRC}/m3-sys/m3cc/${M3TARGET}-${CROSSTARG}/cm3cg \ ${STAGEDIR}${PREFIX}/bin/cm3cg-${CROSSTARG} @(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BOOTCM3} \ ${SH} scripts/do-pkg.sh cleanglobal \ m3middle m3linker m3front m3quake cm3) .for component in ${CORELIBS} @(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BUILTCM3} \ ${SH} scripts/do-pkg.sh buildship ${component:T}) .endfor @(cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} ${BUILTCM3} \ ${SH} scripts/do-pkg.sh cleanglobal ${CORELIBS:T}) .for component in ${CORELIBS} ${COREHELPERS} ${CORESYS} @(cd ${WRKSRC}; \ ${SETENV} ${M3MAKE_ENV} ${BUILTCM3} \ ${SH} scripts/do-pkg.sh buildship ${component:T}; \ ${SETENV} ${M3MAKE_ENV} ${BUILTCM3} \ ${SH} scripts/do-pkg.sh cleanglobal ${component:T} \ ) .endfor ${ECHO} "INSTALL_ROOT = \"${STAGEDIR}${PREFIX}\"" > \ ${STAGEDIR}${PREFIX}/bin/cm3.cfg ${ECHO} "m3back = INSTALL_ROOT & \"/bin/cm3cg-${CROSSTARG}\"" >> \ ${STAGEDIR}${PREFIX}/bin/cm3.cfg ${ECHO} "BUILD_DIR = \"${CROSSTARG}\"" >> \ ${STAGEDIR}${PREFIX}/bin/cm3.cfg ${ECHO} "HOST = \"${M3TARGET}\"" >> ${STAGEDIR}${PREFIX}/bin/cm3.cfg ${ECHO} "include(\"${STAGEDIR}${PREFIX}/etc/modula3/${CROSSTARG}\")" >> \ ${STAGEDIR}${PREFIX}/bin/cm3.cfg ${ECHO} "M3_BOOTSTRAP = TRUE" >> ${STAGEDIR}${PREFIX}/bin/cm3.cfg @${ECHO} "=====================================" @${ECHO} "===== M3 cross-compiler ready =====" @${ECHO} "=====================================" @${ECHO} assembler-code: .for component in ${COREFILES} (cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} SHIP=${TRUE} TARGET=${CROSSTARG} \ ${BUILTCM3} ${SH} scripts/do-pkg.sh onlybuild ${component:T} \ ) .endfor .for component in m3-libs/libm3 m3-sys/m3middle echo '#include "m3core.h"' > \ ${WRKSRC}/${component}/${CROSSTARG}/m3unix.h .endfor ${CP} ${WRKSRC}/m3-libs/m3core/src/m3core.h \ ${WRKSRC}/m3-libs/libm3/${CROSSTARG}/ (cd ${WRKSRC}/m3-sys/m3middle; ${CP} src/POSIX/m3core.h ${CROSSTARG}/) cross-archive: assembler-code (cd ${WRKSRC}; ${SETENV} ${M3MAKE_ENV} M3GDB=no ${BUILTCM3} \ ${SH} scripts/pack-crossbuild.sh ${CROSSTARG}) native-cm3cg: # This target requires python to be installed @(cd ${WRKSRC}/m3-sys/m3cc; src/boot.py) @${MKDIR} ${STAGEDIR}${PREFIX}/bin ${INSTALL_PROGRAM} ${WRKSRC}/m3-sys/m3cc/obj.1/gcc/m3cgc1 \ ${STAGEDIR}${PREFIX}/bin/cm3cg native-link: native-cm3cg .for component in ${COREFILES} @(cd ${WRKSRC}/${CROSSTARG}; ${TAR} -xzf ${component:T}.tgz) .endfor @(cd ${WRKSRC}/scripts; \ ${SETENV} PATH=${PATH}:${STAGEDIR}${PREFIX}/bin \ ./boot-cm3-build-on-target.sh ${CROSSTARG}) ${INSTALL_PROGRAM} ${WRKSRC}/${CROSSTARG}/m3-sys/cm3/${CROSSTARG}/cm3 \ ${STAGEDIR}${PREFIX}/bin ${INSTALL_PROGRAM} ${WRKSRC}/${CROSSTARG}/m3-tools/m3bundle/${CROSSTARG}/m3bundle \ ${STAGEDIR}${PREFIX}/bin From jay.krell at cornell.edu Sun Jan 19 13:17:24 2014 From: jay.krell at cornell.edu (Jay K) Date: Sun, 19 Jan 2014 12:17:24 +0000 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: <52DBC081.1070401@marino.st> References: <52DBA2EF.4020101@marino.st>, , <52DBC081.1070401@marino.st> Message-ID: Hey..I like that that does capture some of My Way. Here is my independent off the cuff write up though: I know the ports system is nice. I know we really need to be in it for many operating systems. I suspect your problem is that you didn't "upgrade" to apply you changes at the right time. Do any of these work for you: https://modula3.elegosoft.com/cm3/snaps/snapshot-index.html Anyway, please try this: Ignore DragonFly at first. Go to a system with a working cm3. Such as FreeBSD. Linux. Almost anything. Such as from ports. Make a backup, e.g. of /usr/local/cm3. My instructions are unfortunately destructive of the existing install. Checkout the full current CVS repository. In that repository, run scripts/python/upgrade.py. If that doesn't work, stop, and we'll help. If that does work, apply your diffs. And upgrade.py again. You can say "upgrade.py skipgcc" here. Then boot1.py c amd64_dragonflybsd Make sure you say "c" and the target "amd64_dragonflybsd", anywhere on the command line. Ok, "c" is actually up to you. Since nobody messes with ABI much, gcc/amd64 works for the same for Linux, FreeBSD, OpenBSD, NetBSD, and Dragonfly. If this works, it will give you, roughly amd64_dragonflybsd*.tar.gz Copy that to your Dragonfly system, extract it, cd into it, make. If that works, you have a native cm3. Run it. It should say "can't find cm3.cfg". Now on Dragonfly, mkdir -p /usr/local/cm3/bin cp cm3 /usr/local/cm3/bin export PATH=/usr/local/cm3/bin:$PATH full CVS checkout cd scripts/python ; ./boot2.sh You will need to have made small changes to pylib.py for this to work. (And your config file should say to use the C backend, like Darwin and ARM_LINUX do.) - Jay > Date: Sun, 19 Jan 2014 13:09:37 +0100 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) > > On 1/19/2014 12:40, Jay K wrote: > > Your change looks about right. > > > > Did you first "upgrade" and existing install, with these changes, and > > then cross? > > That is what you must do. > > > It's probably easier that I show you. > I've attached "Makefile.local" (I added the .txt extension just now to > help out some mail clients) which is my recipe. > > So I run three targets in a freshly patched workarea: > 1) make cross-compiler > 2) make assembler-code (called from cross-archive) > 3) make cross-archive > > In the cross-compiler target, the FreeBSD bootstrap compiler builds > m3middle, m3linker, m3front, mequake, and CM3 for the cross compiler. > > The cross-compiler then builds libm3core and libm3 with FreeBSD cm3cg > Everything is cleaned, then the cross compiler builds everything again > (including libm3core and libm3) along with m3objfile, sysutils, and > patternmatching. > > Then the cm3.cfg is changed to specify HOST=freebsd, TARGET=dragonfly, > cm3cg is cm3cg-AMD64_DRAGONFLY. > > I would not be surprised is there is a logic flaw or two in "make > cross-compiler". The "how to port CM3" tutorial was slightly out of > date and some of the instructions were a bit ambiguous so this was a > best guess. > > Regards, > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Sun Jan 19 13:38:32 2014 From: adacore at marino.st (John Marino) Date: Sun, 19 Jan 2014 13:38:32 +0100 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: References: <52DBA2EF.4020101@marino.st>, , <52DBC081.1070401@marino.st> Message-ID: <52DBC748.5090308@marino.st> On 1/19/2014 13:17, Jay K wrote: > Hey..I like that that does capture some of My Way. > Here is my independent off the cuff write up though: > I know the ports system is nice. > I know we really need to be in it for many operating systems. > I suspect your problem is that you didn't "upgrade" to apply you > changes at the right time. I must be misunderstanding what you mean by "upgrade". It's the cross-compiler that's complaining, and it was built by the latest patches. > Do any of these work for you: > https://modula3.elegosoft.com/cm3/snaps/snapshot-index.html I'm not clear to which set you are referring. There's no need for a binary set, modula3 now builds on FreeBSD 9 and FreeBSD 10 (http://www.freshports.org/lang/modula3), which is also superior to the release binaries because they link gmp and mpfr statically while a bug in the cm3 sources caused those libraries to link dynamically (which is why release binaries don't run on current FreeBSD) Then there are the source distributions, but they are from 2012 so they aren't exactly fresh either. > Anyway, please try this: > Ignore DragonFly at first. > Go to a system with a working cm3. Such as FreeBSD. Linux. Almost > anything. > Such as from ports. > Make a backup, e.g. of /usr/local/cm3. > My instructions are unfortunately destructive of the existing install. > Checkout the full current CVS repository. > In that repository, run scripts/python/upgrade.py. > If that doesn't work, stop, and we'll help. Is the goal just to get FreeBSD to have a binary with the latest sources? If so, I'd just modify the port to do that now that we have working binary bootstraps. The main reason I'm reluctant is that it took over a week to make the cm3 port, I've never had so much difficulty before. (I've also spent another week on the DragonFly port). So I'm "resisting" because of how much work it already took. That's why I was trying to port DragonFly, THEN update to latest rather than update to latest and then port DragonFly. (in other words, going forward seems a short path than backtracking) > If that does work, apply your diffs. And upgrade.py again. You can say > "upgrade.py skipgcc" here. > Then boot1.py c amd64_dragonflybsd > Make sure you say "c" and the target "amd64_dragonflybsd", anywhere > on the command line. > Ok, "c" is actually up to you. Since nobody messes with ABI much, > gcc/amd64 works for the same > for Linux, FreeBSD, OpenBSD, NetBSD, and Dragonfly. I assume you mean after the system understands this target. There will be a lot of files with __FreeBSD__ that need to be changed to __FreeBSD__ || __DragonFly__ as well. In any case, I'll have to regenerate a bunch of patches from my current work. > If this works, it will give you, roughly amd64_dragonflybsd*.tar.gz > Copy that to your Dragonfly system, extract it, cd into it, make. > If that works, you have a native cm3. > Run it. It should say "can't find cm3.cfg". > Now on Dragonfly, mkdir -p /usr/local/cm3/bin > cp cm3 /usr/local/cm3/bin > export PATH=/usr/local/cm3/bin:$PATH > full CVS checkout > cd scripts/python ; ./boot2.sh > > You will need to have made small changes to pylib.py for this to work. > (And your config file should say to use the C backend, like Darwin and > ARM_LINUX do.) Well, I'm thrilled if I don't have to patch gcc again, that takes a long time. Shouldn't FreeBSD use the C backend too? John From jay.krell at cornell.edu Sun Jan 19 13:32:44 2014 From: jay.krell at cornell.edu (Jay K) Date: Sun, 19 Jan 2014 12:32:44 +0000 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: References: <52DBA2EF.4020101@marino.st>, , , , , <52DBC081.1070401@marino.st>, Message-ID: Clarification: upgrade.sh (note the .sh) rebuilds everything I wrote upgrade.py based on it, but missed the "everything" part. upgrade.py only upgrades "the compiler" (and m3core, libm3..) A better practise although inefficient is: upgrade.py && ./do-cm3-all.py realclean skipgcc && ./do-cm3-all.py buildship I didn't suggest that to you here in order to get further in the pipeline faster. However upgrade.py w/o other rebuild could leave m3core.so too new for many programs. But for purposes of building/cross-building just the compiler, my instructions are good. John, I see you added cm3 to the FreeBSD ports and you clearly know what you are doing. Nevertheless, I am a lazy snob, don't quite see what you are doing wrong, and am suggesting "just different", not clearly better. But, to my credit, this "just different" is what I use all the time. I have used it to cross to several systems, w/ and w/o cm3cg (w/ and w/o C backend). It isn't fast, and it doesn't always have the best incrementality, but it does usually work. There is redundancy. pylib.py essentially duplicates parts of the config files. You misunderstood what I meant by "C backend" and from the ports history you will somewhat appreciate it. The C backend, instead of having a lot to do with gcc, is a backend that generates C, and then invokes a C compiler. The problem you had with -gstabs for example, goes away. We'll use whatever debugging format regular C/C++ use. There is a little hope of taking the one C output and compiling it for different targets -- at least FreeBSD 8,9,10. Currently the C is very word size dependent, endian dependent, and somewhat otherwise target-dependent. I'd like to fix these aspects but it isn't easy by far, will take possibly controversial changes in the frontend and interface to backend. - Jay From: jay.krell at cornell.edu To: adacore at marino.st Date: Sun, 19 Jan 2014 12:17:24 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) Hey..I like that that does capture some of My Way. Here is my independent off the cuff write up though: I know the ports system is nice. I know we really need to be in it for many operating systems. I suspect your problem is that you didn't "upgrade" to apply you changes at the right time. Do any of these work for you: https://modula3.elegosoft.com/cm3/snaps/snapshot-index.html Anyway, please try this: Ignore DragonFly at first. Go to a system with a working cm3. Such as FreeBSD. Linux. Almost anything. Such as from ports. Make a backup, e.g. of /usr/local/cm3. My instructions are unfortunately destructive of the existing install. Checkout the full current CVS repository. In that repository, run scripts/python/upgrade.py. If that doesn't work, stop, and we'll help. If that does work, apply your diffs. And upgrade.py again. You can say "upgrade.py skipgcc" here. Then boot1.py c amd64_dragonflybsd Make sure you say "c" and the target "amd64_dragonflybsd", anywhere on the command line. Ok, "c" is actually up to you. Since nobody messes with ABI much, gcc/amd64 works for the same for Linux, FreeBSD, OpenBSD, NetBSD, and Dragonfly. If this works, it will give you, roughly amd64_dragonflybsd*.tar.gz Copy that to your Dragonfly system, extract it, cd into it, make. If that works, you have a native cm3. Run it. It should say "can't find cm3.cfg". Now on Dragonfly, mkdir -p /usr/local/cm3/bin cp cm3 /usr/local/cm3/bin export PATH=/usr/local/cm3/bin:$PATH full CVS checkout cd scripts/python ; ./boot2.sh You will need to have made small changes to pylib.py for this to work. (And your config file should say to use the C backend, like Darwin and ARM_LINUX do.) - Jay > Date: Sun, 19 Jan 2014 13:09:37 +0100 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) > > On 1/19/2014 12:40, Jay K wrote: > > Your change looks about right. > > > > Did you first "upgrade" and existing install, with these changes, and > > then cross? > > That is what you must do. > > > It's probably easier that I show you. > I've attached "Makefile.local" (I added the .txt extension just now to > help out some mail clients) which is my recipe. > > So I run three targets in a freshly patched workarea: > 1) make cross-compiler > 2) make assembler-code (called from cross-archive) > 3) make cross-archive > > In the cross-compiler target, the FreeBSD bootstrap compiler builds > m3middle, m3linker, m3front, mequake, and CM3 for the cross compiler. > > The cross-compiler then builds libm3core and libm3 with FreeBSD cm3cg > Everything is cleaned, then the cross compiler builds everything again > (including libm3core and libm3) along with m3objfile, sysutils, and > patternmatching. > > Then the cm3.cfg is changed to specify HOST=freebsd, TARGET=dragonfly, > cm3cg is cm3cg-AMD64_DRAGONFLY. > > I would not be surprised is there is a logic flaw or two in "make > cross-compiler". The "how to port CM3" tutorial was slightly out of > date and some of the instructions were a bit ambiguous so this was a > best guess. > > Regards, > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Jan 19 13:50:27 2014 From: jay.krell at cornell.edu (Jay K) Date: Sun, 19 Jan 2014 12:50:27 +0000 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: <52DBC748.5090308@marino.st> References: <52DBA2EF.4020101@marino.st>, , <52DBC081.1070401@marino.st> , <52DBC748.5090308@marino.st> Message-ID: I understand your reluctance. > Is the goal just to get FreeBSD to have a binary with the latest > sources? Yes, and to make sure the build process works for you. > Shouldn't FreeBSD use the C backend too? Every target should. I haven't convinced everyone yet. There is less resistance for new targets, like AMD64_NT and ARM_LINUX, which are working, using the C backend. There are downsides to the C backend: - it is slower to compiler - debugging is degraded on systems that have m3gdb (debugging is much better on systems w/o m3gdb) And there are advantages not yet implemented, in particular, efficient exception handling by generating C++. > > gcc/amd64 works for the same > > for Linux, FreeBSD, OpenBSD, NetBSD, and Dragonfly. > > I assume you mean after the system understands this target. There will > be a lot of files with __FreeBSD__ that need to be changed to > __FreeBSD__ || __DragonFly__ as well. In any case, I'll have to > regenerate a bunch of patches from my current work. Your patches should apply fairly cleanly to any new checkout. I am willing to through them and apply them to CVS if you want -- depends on if you are ok with my stealing your credit, vs. you want to commit them yourselves. I already skimmed them and they all look easy. DragonflyBSD is basically FreeBSD by another name. Yes, a ton of work has been done on the kernel, file systems, maybe ports. But the ABI is almost the same -- heck, for the most part, Linux==NetBSD==FreeBSD==OpenBSD. Sure, they might be implemented differently, but they are highly source compatible and every highly object code compatible. > I must be misunderstanding what you mean by "upgrade". It's the > cross-compiler that's complaining, and it was built by the latest patches. Agreed, we are both missing something simple. Usually this is the error you get when you don't upgrade Target.i3/Target.m3. If you want to press on with the current approach and ignore my advise, it should be possible to step through all this. To start, break on Target__Init and see if it returns true or false. > statically while a bug in the cm3 sources caused those libraries to link > dynamically (which is why release binaries don't run on current FreeBSD) Ugh. I saw something about that in your diffs. Definitely we intend to link statically. I actually went to the trouble of removing use of gmp/mpfr/mpc in current source. They are used for actually very little and aren't worth it. - Jay > Date: Sun, 19 Jan 2014 13:38:32 +0100 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) > > On 1/19/2014 13:17, Jay K wrote: > > Hey..I like that that does capture some of My Way. > > Here is my independent off the cuff write up though: > > I know the ports system is nice. > > I know we really need to be in it for many operating systems. > > I suspect your problem is that you didn't "upgrade" to apply you > > changes at the right time. > > I must be misunderstanding what you mean by "upgrade". It's the > cross-compiler that's complaining, and it was built by the latest patches. > > > > Do any of these work for you: > > https://modula3.elegosoft.com/cm3/snaps/snapshot-index.html > > I'm not clear to which set you are referring. > There's no need for a binary set, modula3 now builds on FreeBSD 9 and > FreeBSD 10 (http://www.freshports.org/lang/modula3), which is also > superior to the release binaries because they link gmp and mpfr > statically while a bug in the cm3 sources caused those libraries to link > dynamically (which is why release binaries don't run on current FreeBSD) > > Then there are the source distributions, but they are from 2012 so they > aren't exactly fresh either. > > > > Anyway, please try this: > > Ignore DragonFly at first. > > Go to a system with a working cm3. Such as FreeBSD. Linux. Almost > > anything. > > Such as from ports. > > Make a backup, e.g. of /usr/local/cm3. > > My instructions are unfortunately destructive of the existing install. > > Checkout the full current CVS repository. > > In that repository, run scripts/python/upgrade.py. > > If that doesn't work, stop, and we'll help. > > Is the goal just to get FreeBSD to have a binary with the latest > sources? If so, I'd just modify the port to do that now that we have > working binary bootstraps. The main reason I'm reluctant is that it > took over a week to make the cm3 port, I've never had so much difficulty > before. (I've also spent another week on the DragonFly port). So I'm > "resisting" because of how much work it already took. That's why I was > trying to port DragonFly, THEN update to latest rather than update to > latest and then port DragonFly. (in other words, going forward seems a > short path than backtracking) > > > > > If that does work, apply your diffs. And upgrade.py again. You can say > > "upgrade.py skipgcc" here. > > Then boot1.py c amd64_dragonflybsd > > Make sure you say "c" and the target "amd64_dragonflybsd", anywhere > > on the command line. > > Ok, "c" is actually up to you. Since nobody messes with ABI much, > > gcc/amd64 works for the same > > for Linux, FreeBSD, OpenBSD, NetBSD, and Dragonfly. > > I assume you mean after the system understands this target. There will > be a lot of files with __FreeBSD__ that need to be changed to > __FreeBSD__ || __DragonFly__ as well. In any case, I'll have to > regenerate a bunch of patches from my current work. > > > > If this works, it will give you, roughly amd64_dragonflybsd*.tar.gz > > Copy that to your Dragonfly system, extract it, cd into it, make. > > If that works, you have a native cm3. > > Run it. It should say "can't find cm3.cfg". > > Now on Dragonfly, mkdir -p /usr/local/cm3/bin > > cp cm3 /usr/local/cm3/bin > > export PATH=/usr/local/cm3/bin:$PATH > > full CVS checkout > > cd scripts/python ; ./boot2.sh > > > > You will need to have made small changes to pylib.py for this to work. > > (And your config file should say to use the C backend, like Darwin and > > ARM_LINUX do.) > > > Well, I'm thrilled if I don't have to patch gcc again, that takes a long > time. Shouldn't FreeBSD use the C backend too? > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Sun Jan 19 14:06:02 2014 From: adacore at marino.st (John Marino) Date: Sun, 19 Jan 2014 14:06:02 +0100 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: References: <52DBA2EF.4020101@marino.st>, , <52DBC081.1070401@marino.st> , <52DBC748.5090308@marino.st> Message-ID: <52DBCDBA.8030809@marino.st> On 1/19/2014 13:50, Jay K wrote: > And there are advantages not yet implemented, in particular, efficient > exception handling by generating C++. Like FreeBSD (and unlike NetBSD and OpenBSD), DragonFly has implemented dl_iterate_phdr which gcc uses for efficient zero-cost exception handling. That might come in handy here (I implemented it in DragonFly's real-time linker, mainly for Ada/GNAT but C++ benefited.) > Your patches should apply fairly cleanly to any new checkout. > I am willing to through them and apply them to CVS if you want -- > depends on if you are ok with my stealing your credit, vs. you want to > commit them yourselves. I would love it if you did this - I'm happy with a mention in the commit message. I just attached a tarball of all the patches I'm using right now. Some of these changes are specifically for the cross compiler (I'm think of the /scripts changes mostly) but most are valid assuming they still apply to the head of the repo. Please take as much as you want with my gratitude. > I already skimmed them and they all look easy. > DragonflyBSD is basically FreeBSD by another name. That's not an accident. As a DragonFly developer that mostly works on userland, I've been converging FreeBSD and DragonFly again. They drifted apart after 10 years, but I thought it better to keep the userlands as similar as possible for 3rd party applications. > Yes, a ton of work has been done on the kernel, file systems, maybe ports. > But the ABI is almost the same -- heck, for the most part, > Linux==NetBSD==FreeBSD==OpenBSD. > Sure, they might be implemented differently, but they are highly source > compatible and every highly object code compatible. >> I must be misunderstanding what you mean by "upgrade". It's the >> cross-compiler that's complaining, and it was built by the latest patches. > Agreed, we are both missing something simple. > Usually this is the error you get when you don't upgrade > Target.i3/Target.m3. > If you want to press on with the current approach and ignore my advise, I'm not ignoring it; it's more like I'm not quite ready to give up since I'm so close. I think I could actually get these libs and cm3 to build on DragonFly if I used AMD64_FREEBSD as the target with the cm3cg-AMD_DRAGONFLY generator. It was building before. I just figured I'd then get stuck on the DragonFly side trying to compile cm3 with itself, theoretically I should get the same error there. > Ugh. I saw something about that in your diffs. Definitely we intend to > link statically. > I actually went to the trouble of removing use of gmp/mpfr/mpc in > current source. > They are used for actually very little and aren't worth it. Even better! Regards, John -------------- next part -------------- A non-text attachment was scrubbed... Name: files.tar Type: application/x-tar Size: 13062 bytes Desc: not available URL: From dragisha at m3w.org Sun Jan 19 15:28:18 2014 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 19 Jan 2014 15:28:18 +0100 Subject: [M3devel] Cooperative suspend? In-Reply-To: References: <8BC1682A-3760-44D0-BA67-3A0F3EA10E27@m3w.org> Message-ID: Why would we do this, and not signalling, as in POSIX implementation? Why try to make something synchronous when we have working asynchronous implementation? Why would it be better than sync one? On 19 Jan 2014, at 13:00, Jay K wrote: > There is a list of threads maintained by the m3 runtime. > Not by enumerating OS threads. > The thread structure contains in it perhaps a boolean or enum as to suspend requested, suspending, suspended, resuming, running, etc. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mika at async.caltech.edu Sun Jan 19 21:05:48 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sun, 19 Jan 2014 12:05:48 -0800 Subject: [M3devel] Win32 SuspendThread? In-Reply-To: <9EC3F463-F248-4C79-AF3F-130D2252B8B0@gmail.com> References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL> <9EC3F463-F248-4C79-AF3F-130D2252B8B0@gmail.com> Message-ID: <20140119200548.C18011A208F@async.async.caltech.edu> "less worried" = "I think it works"? I think the only threading that works in CM3 is user threading... I tried to run CM3-compiled binaries through "valgrind" but got an error that CM3 is using the same signal that valgrind does internally (on AMD64_LINUX at least). I think this is easy to change in m3core (yes?) but haven't gotten around to it... valgrind has a threading checker called "helgrind"... Mika Jay writes: > >--Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A >Content-Type: text/plain; > charset=utf-8 >Content-Transfer-Encoding: quoted-printable > >For native I'm less worried now. For wow64 I'm still worried & hope to follo= >w up further. > > - Jay > >On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" wrote:= > > >> Jay: >> =20 >> I=E2=80=99m very concerned about the threading not working properly on bot= >h 32-bit and 64-bit Windows. >> =20 >> The Thread Test program crashes for me on both platforms. >> =20 >> I haven=E2=80=99t tried your new test program yet. >> =20 >> --Randy >> =20 >> From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K >> Sent: Saturday, January 18, 2014 1:23 AM >> To: m3devel >> Subject: EXT:[M3devel] Win32 SuspendThread? >> =20 >> This program also doesn't behave as expected, native, nothing to do with w= >ow64.=20 >> Anyone else please confirm:=20 >> 1) my expectations -- it should never print anything >> 2) their importance -- garbage collector depends on it =20 >> 3) their not being met -- stuff gets printed=20 >> This is in the CVS repository, scratch/wow64stack/sync2.cpp=20 >> I am following up further.=20 >> Maybe we should get cooperative suspend really going?=20 >> Thank you. >> - Jay=20 >> =20 >> #include >> #include >> volatile long value; >> unsigned long __stdcall Thread(PVOID parameter) >> { >> while (1) >> InterlockedIncrement(&value); >> return 0; >> } >> int __cdecl main() >> { >> HANDLE thread =3D CreateThread(0, 0, Thread, 0, 0, 0); >> UINT i =3D 0; >> while (1) >> { >> i +=3D 1; >> if (SuspendThread(thread) =3D=3D (DWORD)-1) >> { >> printf("suspend failed %X\n", GetLastError()); >> Sleep(1); >> continue; >> } >> volatile long a =3D value;=20 >> volatile long b =3D value; >> if (a !=3D b) >> { >> printf("%d %d %d %d\n", i, a, b, b - a); >> } >> ResumeThread(thread); >> } >> } > >--Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A >Content-Type: text/html; > charset=utf-8 >Content-Transfer-Encoding: quoted-printable > >utf-8">
For native I'm less worried now. For w= >ow64 I'm still worried & hope to follow up further.

 - Jaydiv>

On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" <mailto:rcolebur at SCIRES.COM">rcolebur at SCIRES.COM> wrote:

= >
> >= > > > > > >
>

ibri","sans-serif";color:#1F497D">Jay:

>

ibri","sans-serif";color:#1F497D"> > >

ibri","sans-serif";color:#1F497D">I=E2=80=99m very concerned a= >bout the threading not working properly on both 32-bit and 64-bit Windows.:p>

>

ibri","sans-serif";color:#1F497D"> > >

ibri","sans-serif";color:#1F497D">The Thread Test program cra= >shes for me on both platforms.

>

ibri","sans-serif";color:#1F497D"> > >

ibri","sans-serif";color:#1F497D">I haven=E2=80=99t tried you= >r new test program yet.

>

ibri","sans-serif";color:#1F497D"> > >

ibri","sans-serif";color:#1F497D">--Randyp> >

ibri","sans-serif";color:#1F497D"> > >

>
in 0in"> >

e:10.0pt;font-family:"Tahoma","sans-serif"">From:= >s-serif""> jayk123 at hotmail.coma> [mailto:jayk123 at hotmail.com] >On Behalf Of Jay K
>Sent: Saturday, January 18, 2014 1:23 AM
>To: m3devel
>Subject: EXT:[M3devel] Win32 SuspendThread?

>
>
>

 

>
>

in-bottom:12.0pt;margin-left:.5in"> > = >;This program also doesn't behave as expected, native, nothing to do with wo= >w64. >
> Anyone else please confirm:
>   1) my expectations -- it should never print anything
>   2) their importance  -- garbage collector depends on it&nb= >sp;
>   3) their not being met -- stuff gets printed
> This is in the CVS repository, scratch/wow64stack/sync2.cpp
> I am following up further.
> Maybe we should get cooperative suspend really going?
>Thank you.
> - Jay

>#include <stdio.h>
>#include <windows.h>
>volatile long value;
>unsigned long __stdcall Thread(PVOID parameter)
>{
>  while (1)
>   InterlockedIncrement(&value);
>  return 0;
>}
>int __cdecl main()
>{
>  HANDLE thread =3D CreateThread(0, 0, Thread, 0, 0, 0);
>  UINT i =3D 0;
>  while (1)
>  {
>    i +=3D 1;
>    if (SuspendThread(thread) =3D=3D (DWORD)-1)
>    {
>      printf("suspend failed %X\n", GetLastError())= >;
>      Sleep(1);
>      continue;
>    }
>    volatile long a =3D value;
>    volatile long b =3D value;
>    if (a !=3D b)
>    {
>     printf("%d %d %d %d\n", i, a, b, b - a);
>    }
>    ResumeThread(thread);
>  }
>}

>
>
> > >
= > >--Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A-- From mika at async.caltech.edu Sun Jan 19 21:13:08 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sun, 19 Jan 2014 12:13:08 -0800 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: References: <52DBA2EF.4020101@marino.st>, , <52DBC081.1070401@marino.st> , <52DBC748.5090308@marino.st> Message-ID: <20140119201308.0A4B41A208F@async.async.caltech.edu> There were some bugs we never got around to addressing with the C backend, weren't there? A crash that didn't happen with native backend. I've been using the "native" at more or less the head on very recent Debian and the Raspberry Pi for a while now with no problems to speak of. (As long as you only use user threads... but that's another story, I think.) Mika Jay K writes: >--_f56ed036-02b3-494e-bb04-f7c58d07fa28_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > >I understand your reluctance. > > >> Is the goal just to get FreeBSD to have a binary with the latest >> sources? > > >Yes=2C and to make sure the build process works for you. > > >> Shouldn't FreeBSD use the C backend too? > > >Every target should. I haven't convinced everyone yet. >There is less resistance for new targets=2C like AMD64_NT and ARM_LINUX=2C = >which are working=2C using the C backend. >There are downsides to the C backend: > - it is slower to compiler =20 > - debugging is degraded on systems that have m3gdb =20 > (debugging is much better on systems w/o m3gdb)=20 > > >And there are advantages not yet implemented=2C in particular=2C efficient = >exception handling by generating C++. > > >> > gcc/amd64 works for the same >> > for Linux=2C FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly. >>=20 >> I assume you mean after the system understands this target. There will >> be a lot of files with __FreeBSD__ that need to be changed to >> __FreeBSD__ || __DragonFly__ as well. In any case=2C I'll have to >> regenerate a bunch of patches from my current work. > > >Your patches should apply fairly cleanly to any new checkout. >I am willing to through them and apply them to CVS if you want -- depends o= >n if you are ok with my stealing your credit=2C vs. you want to commit them= > yourselves. >I already skimmed them and they all look easy. >DragonflyBSD is basically FreeBSD by another name. >Yes=2C a ton of work has been done on the kernel=2C file systems=2C maybe p= >orts. >But the ABI is almost the same -- heck=2C for the most part=2C Linux=3D=3DN= >etBSD=3D=3DFreeBSD=3D=3DOpenBSD. >Sure=2C they might be implemented differently=2C but they are highly source= > compatible and every highly object code compatible. > > >> I must be misunderstanding what you mean by "upgrade". It's the >> cross-compiler that's complaining=2C and it was built by the latest patch= >es. > >Agreed=2C we are both missing something simple. >Usually this is the error you get when you don't upgrade Target.i3/Target.m= >3. >If you want to press on with the current approach and ignore my advise=2C i= >t should be possible to step through all this. >To start=2C break on Target__Init and see if it returns true or false. > > >> statically while a bug in the cm3 sources caused those libraries to link >> dynamically (which is why release binaries don't run on current FreeBSD) > > >Ugh. I saw something about that in your diffs. Definitely we intend to link= > statically. >I actually went to the trouble of removing use of gmp/mpfr/mpc in current s= >ource. >They are used for actually very little and aren't worth it. > > > - Jay > > >> Date: Sun=2C 19 Jan 2014 13:38:32 +0100 >> From: adacore at marino.st >> To: jay.krell at cornell.edu >> CC: m3devel at elegosoft.com >> Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) >>=20 >> On 1/19/2014 13:17=2C Jay K wrote: >> > Hey..I like that that does capture some of My Way.=20 >> > Here is my independent off the cuff write up though: >> > I know the ports system is nice. >> > I know we really need to be in it for many operating systems. >> > I suspect your problem is that you didn't "upgrade" to apply you >> > changes at the right time. >>=20 >> I must be misunderstanding what you mean by "upgrade". It's the >> cross-compiler that's complaining=2C and it was built by the latest patch= >es. >>=20 >>=20 >> > Do any of these work for you: >> > https://modula3.elegosoft.com/cm3/snaps/snapshot-index.html >>=20 >> I'm not clear to which set you are referring. >> There's no need for a binary set=2C modula3 now builds on FreeBSD 9 and >> FreeBSD 10 (http://www.freshports.org/lang/modula3)=2C which is also >> superior to the release binaries because they link gmp and mpfr >> statically while a bug in the cm3 sources caused those libraries to link >> dynamically (which is why release binaries don't run on current FreeBSD) >>=20 >> Then there are the source distributions=2C but they are from 2012 so they >> aren't exactly fresh either. >>=20 >>=20 >> > Anyway=2C please try this:=20 >> > Ignore DragonFly at first.=20 >> > Go to a system with a working cm3. Such as FreeBSD. Linux. Almost >> > anything.=20 >> > Such as from ports.=20 >> > Make a backup=2C e.g. of /usr/local/cm3.=20 >> > My instructions are unfortunately destructive of the existing install= >.=20 >> > Checkout the full current CVS repository.=20 >> > In that repository=2C run scripts/python/upgrade.py.=20 >> > If that doesn't work=2C stop=2C and we'll help.=20 >>=20 >> Is the goal just to get FreeBSD to have a binary with the latest >> sources? If so=2C I'd just modify the port to do that now that we have >> working binary bootstraps. The main reason I'm reluctant is that it >> took over a week to make the cm3 port=2C I've never had so much difficult= >y >> before. (I've also spent another week on the DragonFly port). So I'm >> "resisting" because of how much work it already took. That's why I was >> trying to port DragonFly=2C THEN update to latest rather than update to >> latest and then port DragonFly. (in other words=2C going forward seems a >> short path than backtracking) >>=20 >>=20 >>=20 >> > If that does work=2C apply your diffs. And upgrade.py again. You can = >say >> > "upgrade.py skipgcc" here.=20 >> > Then boot1.py c amd64_dragonflybsd =20 >> > Make sure you say "c" and the target "amd64_dragonflybsd"=2C anywhe= >re >> > on the command line.=20 >> > Ok=2C "c" is actually up to you. Since nobody messes with ABI much= >=2C >> > gcc/amd64 works for the same >> > for Linux=2C FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly. >>=20 >> I assume you mean after the system understands this target. There will >> be a lot of files with __FreeBSD__ that need to be changed to >> __FreeBSD__ || __DragonFly__ as well. In any case=2C I'll have to >> regenerate a bunch of patches from my current work. >>=20 >>=20 >> > If this works=2C it will give you=2C roughly amd64_dragonflybsd*.tar.= >gz=20 >> > Copy that to your Dragonfly system=2C extract it=2C cd into it=2C mak= >e.=20 >> > If that works=2C you have a native cm3.=20 >> > Run it. It should say "can't find cm3.cfg". =20 >> > Now on Dragonfly=2C mkdir -p /usr/local/cm3/bin=20 >> > cp cm3 /usr/local/cm3/bin =20 >> > export PATH=3D/usr/local/cm3/bin:$PATH >> > full CVS checkout =20 >> > cd scripts/python =3B ./boot2.sh=20 >> > =20 >> > You will need to have made small changes to pylib.py for this to work= >. >> > (And your config file should say to use the C backend=2C like Darwin = >and >> > ARM_LINUX do.) >>=20 >>=20 >> Well=2C I'm thrilled if I don't have to patch gcc again=2C that takes a l= >ong >> time. Shouldn't FreeBSD use the C backend too? >>=20 >> John > = > >--_f56ed036-02b3-494e-bb04-f7c58d07fa28_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > >
I understand your reluctance.>

>=3B Is the goal just to get FreeBSD to have a binary with the l= >atest
>=3B sources?


Yes=2C and to make sure the build proce= >ss works for you.


>=3B Shouldn't FreeBSD use the C backend too= >?


Every target should. I haven't convinced everyone yet.
Ther= >e is less resistance for new targets=2C like AMD64_NT and ARM_LINUX=2C whic= >h are working=2C using the C backend.
There are downsides to the C backe= >nd:
 =3B - it is slower to compiler =3B
 =3B - debugging= > is degraded on systems that have m3gdb =3B
 =3B(debugging is m= >uch better on systems w/o m3gdb)


And there are advantages not y= >et implemented=2C in particular=2C efficient exception handling by generati= >ng C++.


>=3B >=3B gcc/amd64 works for the same
>=3B >= >=3B for Linux=2C FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.
>= >=3B
>=3B I assume you mean after the system understands this target. = > There will
>=3B be a lot of files with __FreeBSD__ that need to be ch= >anged to
>=3B __FreeBSD__ || __DragonFly__ as well. In any case=2C I'= >ll have to
>=3B regenerate a bunch of patches from my current work.>

Your patches should apply fairly cleanly to any new checkout.
I= > am willing to through them and apply them to CVS if you want -- depends on= > if you are ok with my stealing your credit=2C vs. you want to commit them = >yourselves.
I already skimmed them and they all look easy.
DragonflyB= >SD is basically FreeBSD by another name.
Yes=2C a ton of work has been d= >one on the kernel=2C file systems=2C maybe ports.
But the ABI is almost = >the same -- heck=2C for the most part=2C Linux=3D=3DNetBSD=3D=3DFreeBSD=3D= >=3DOpenBSD.
Sure=2C they might be implemented differently=2C but they ar= >e highly source compatible and every highly object code compatible.

= >
>=3B I must be misunderstanding what you mean by "upgrade". It's the= >
>=3B cross-compiler that's complaining=2C and it was built by the lat= >est patches.

Agreed=2C we are both missing something simple.
Usua= >lly this is the error you get when you don't upgrade Target.i3/Target.m3.r>If you want to press on with the current approach and ignore my advise=2C= > it should be possible to step through all this.
To start=2C break on Ta= >rget__Init and see if it returns true or false.


>=3B staticall= >y while a bug in the cm3 sources caused those libraries to link
>=3B d= >ynamically (which is why release binaries don't run on current FreeBSD)
= >

Ugh. I saw something about that in your diffs. Definitely we intend= > to link statically.
I actually went to the trouble of removing use of g= >mp/mpfr/mpc in current source.
They are used for actually very little an= >d aren't worth it.


 =3B- Jay


>=3B Date: Su= >n=2C 19 Jan 2014 13:38:32 +0100
>=3B From: adacore at marino.st
>=3B= > To: jay.krell at cornell.edu
>=3B CC: m3devel at elegosoft.com
>=3B Su= >bject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
&g= >t=3B
>=3B On 1/19/2014 13:17=2C Jay K wrote:
>=3B >=3B Hey..I = >like that that does capture some of My Way.
>=3B >=3B Here is my in= >dependent off the cuff write up though:
>=3B >=3B I know the ports = >system is nice.
>=3B >=3B I know we really need to be in it for man= >y operating systems.
>=3B >=3B I suspect your problem is that you d= >idn't "upgrade" to apply you
>=3B >=3B changes at the right time.>>=3B
>=3B I must be misunderstanding what you mean by "upgrade". = >It's the
>=3B cross-compiler that's complaining=2C and it was built by= > the latest patches.
>=3B
>=3B
>=3B >=3B Do any of thes= >e work for you:
>=3B >=3B https://modula3.elegosoft.com/cm3/snaps/s= >napshot-index.html
>=3B
>=3B I'm not clear to which set you are = >referring.
>=3B There's no need for a binary set=2C modula3 now builds= > on FreeBSD 9 and
>=3B FreeBSD 10 (http://www.freshports.org/lang/modu= >la3)=2C which is also
>=3B superior to the release binaries because th= >ey link gmp and mpfr
>=3B statically while a bug in the cm3 sources ca= >used those libraries to link
>=3B dynamically (which is why release bi= >naries don't run on current FreeBSD)
>=3B
>=3B Then there are th= >e source distributions=2C but they are from 2012 so they
>=3B aren't e= >xactly fresh either.
>=3B
>=3B
>=3B >=3B Anyway=2C plea= >se try this:
>=3B >=3B Ignore DragonFly at first.
>=3B >= >=3B Go to a system with a working cm3. Such as FreeBSD. Linux. Almost
= >>=3B >=3B anything.
>=3B >=3B Such as from ports.
>=3B = >>=3B Make a backup=2C e.g. of /usr/local/cm3.
>=3B >=3B My in= >structions are unfortunately destructive of the existing install.
>= >=3B >=3B Checkout the full current CVS repository.
>=3B >=3B = >In that repository=2C run scripts/python/upgrade.py.
>=3B >=3B If= > that doesn't work=2C stop=2C and we'll help.
>=3B
>=3B Is the = >goal just to get FreeBSD to have a binary with the latest
>=3B sources= >? If so=2C I'd just modify the port to do that now that we have
>=3B = >working binary bootstraps. The main reason I'm reluctant is that it
>= >=3B took over a week to make the cm3 port=2C I've never had so much difficu= >lty
>=3B before. (I've also spent another week on the DragonFly port)= >. So I'm
>=3B "resisting" because of how much work it already took. = >That's why I was
>=3B trying to port DragonFly=2C THEN update to lates= >t rather than update to
>=3B latest and then port DragonFly. (in othe= >r words=2C going forward seems a
>=3B short path than backtracking)>>=3B
>=3B
>=3B
>=3B >=3B If that does work=2C appl= >y your diffs. And upgrade.py again. You can say
>=3B >=3B "upgrade.p= >y skipgcc" here.
>=3B >=3B Then boot1.py c amd64_dragonflybsd <= >br>>=3B >=3B Make sure you say "c" and the target "amd64_dragonflyb= >sd"=2C anywhere
>=3B >=3B on the command line.
>=3B >=3B = > Ok=2C "c" is actually up to you. Since nobody messes with ABI much=2C
&= >gt=3B >=3B gcc/amd64 works for the same
>=3B >=3B for Linux=2C= > FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.
>=3B
>=3B I assu= >me you mean after the system understands this target. There will
>=3B= > be a lot of files with __FreeBSD__ that need to be changed to
>=3B __= >FreeBSD__ || __DragonFly__ as well. In any case=2C I'll have to
>=3B = >regenerate a bunch of patches from my current work.
>=3B
>=3B r>>=3B >=3B If this works=2C it will give you=2C roughly amd64_dragon= >flybsd*.tar.gz
>=3B >=3B Copy that to your Dragonfly system=2C ex= >tract it=2C cd into it=2C make.
>=3B >=3B If that works=2C you ha= >ve a native cm3.
>=3B >=3B Run it. It should say "can't find cm3.= >cfg".
>=3B >=3B Now on Dragonfly=2C mkdir -p /usr/local/cm3/bin = >
>=3B >=3B cp cm3 /usr/local/cm3/bin
>=3B >=3B export P= >ATH=3D/usr/local/cm3/bin:$PATH
>=3B >=3B full CVS checkout
&g= >t=3B >=3B cd scripts/python =3B ./boot2.sh
>=3B >=3B
>= >=3B >=3B You will need to have made small changes to pylib.py for this = >to work.
>=3B >=3B (And your config file should say to use the C b= >ackend=2C like Darwin and
>=3B >=3B ARM_LINUX do.)
>=3B
>= >=3B
>=3B Well=2C I'm thrilled if I don't have to patch gcc again=2C t= >hat takes a long
>=3B time. Shouldn't FreeBSD use the C backend too?<= >br>>=3B
>=3B John
>= > >--_f56ed036-02b3-494e-bb04-f7c58d07fa28_-- From mika at async.caltech.edu Sun Jan 19 22:36:49 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sun, 19 Jan 2014 13:36:49 -0800 Subject: [M3devel] narrow still failing.. Message-ID: <20140119213649.403AE1A208F@async.async.caltech.edu> A smaller test: 1 MODULE Main; 2 IMPORT BDD, IO; 3 4 PROCEDURE P() : REFANY = 5 BEGIN 6 VAR x : REFANY; 7 BEGIN 8 x := BDD.New("a"); 9 Z(x); 10 RETURN x 11 END 12 END P; 13 14 PROCEDURE Q() = 15 BEGIN 16 IO.Put(P() & "\n") 17 END Q; 18 19 PROCEDURE Z(VAR x : REFANY) = 20 BEGIN 21 x := BDD.Format(x) 22 END Z; 23 24 BEGIN 25 Q() 26 END Main. (191)rover:~/refany/src>../FreeBSD4/refany *** *** runtime error: *** NARROW failed *** file "/big/home/mika/refany/src/Main.m3", line 21 *** What's going on here??? Here are the relevant declarations: (* simple BDD package *) INTERFACE BDD; IMPORT Word; TYPE T <: REFANY; (* the boolean constants top and bottom *) PROCEDURE True() : T; PROCEDURE False() : T; (* a new variable *) PROCEDURE New(name : TEXT := NIL) : T; (* unary ops *) PROCEDURE Not(a : T) : T; (* binary ops *) PROCEDURE And(a, b : T) : T; PROCEDURE Xor(a , b : T) : T; PROCEDURE Equivalent(a, b : T) : T; PROCEDURE Or( a, b : T) : T; PROCEDURE Implies (a , b : T) : T; (* maketrue/makefalse *) PROCEDURE MakeFalse(b, v : T) : T; (* make v false in b *) PROCEDURE MakeTrue(b, v : T) : T; (* make v true in b *) (* print with ids *) PROCEDURE Format(a : T; symtab : REFANY (* BDDTextTbl.T *) := NIL) : TEXT; (* the following procedures allow this interface to be used in generics *) PROCEDURE Equal(a, b : T) : BOOLEAN; PROCEDURE Hash(a : T) : Word.T; PROCEDURE Size(a : T) : CARDINAL; (* number of nodes in structure *) CONST Brand = "BDD 0.1"; PROCEDURE GetId(a : T) : INTEGER; (* for debugging *) END BDD. .... REVEAL T = BRANDED Brand OBJECT l , r : T; root : Root; tag : CARDINAL; (* for hashing *) name : TEXT; METHODS init() : T := Init; END; PROCEDURE New(name : TEXT) : T = VAR res : Root := NEW(Root).init(); BEGIN res.name := name; res.root := res; res.l := true; res.r := false; res.id := nextId; res.tab := NEW(BDDTripleHash.Default).init(128); FOR i := FIRST(res.cache) TO LAST(res.cache) DO res.cache[i] := NEW(BDDTripleHash.Default).init(64) END; EVAL BDDTripleHash.Put(res.tab, Pair { res.l, res.r }, (res)); INC(nextId); RETURN res END New; PROCEDURE Format(x : T; symtab : REFANY := NIL) : TEXT = VAR nm : TEXT; BEGIN IF symtab # NIL AND NARROW(symtab, BDDTextTbl.T).get(x, nm) THEN RETURN nm END; IF x = true THEN RETURN "TRUE" ELSIF x = false THEN RETURN "FALSE" END; IF x.name # NIL THEN RETURN x.name END; RETURN Fmt.Int(x.root.id) & " && (" & Format(x.l) & ") || (" & Format(x.r) & ") && ~" & Fmt.Int(x.root.id) END Format; use option @M3stackdump to get a stack trace Abort I'm happy to share the whole package if anyone with a better clue than me would like a look. In fact: http://rover.gcapltd.com/~mika/bdd.tgz It should build without anything outside of standard cm3. Mika From mika at async.caltech.edu Sun Jan 19 23:01:23 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sun, 19 Jan 2014 14:01:23 -0800 Subject: [M3devel] narrow still failing.. In-Reply-To: <20140119213649.403AE1A208F@async.async.caltech.edu> References: <20140119213649.403AE1A208F@async.async.caltech.edu> Message-ID: <20140119220123.0D1971A208F@async.async.caltech.edu> As in the ISTYPE/TYPECASE example, the following works: PROCEDURE Z(VAR x : REFANY) = BEGIN TYPECASE x OF BDD.T(b) => x := BDD.Format(b) END; END Z; (244)rover:~/refany/src>../FreeBSD4/refany a This does NOT work: PROCEDURE Z(VAR x : REFANY) = BEGIN TYPECASE x OF BDD.T(b) => x := BDD.Format(x) END; END Z; (242)rover:~/refany/src>../FreeBSD4/refany *** *** runtime error: *** NARROW failed *** file "/big/home/mika/refany/src/Main.m3", line 22 *** use option @M3stackdump to get a stack trace Abort Same result PM3 / CM3. Mika From jay.krell at cornell.edu Sun Jan 19 23:58:27 2014 From: jay.krell at cornell.edu (Jay K) Date: Sun, 19 Jan 2014 22:58:27 +0000 Subject: [M3devel] Cooperative suspend? In-Reply-To: References: <8BC1682A-3760-44D0-BA67-3A0F3EA10E27@m3w.org> , Message-ID: 1) portability; We currently have separate code, roughly, for MacOSX, FreeBSD, OpenBSD, NT, and the other Posixes. MacOSX doesn't have Posix semaphores, for example. This would give us one codebase here. Only IA64 would likely remain an outlier, because stack is not merely the address of one local to another, but includes also the grow-up stack for the register saves. 2) It'd workaround the wow64 problem. ?- Jay ________________________________ > Subject: Re: [M3devel] Cooperative suspend? > From: dragisha at m3w.org > Date: Sun, 19 Jan 2014 15:28:18 +0100 > CC: m3devel at elegosoft.com > To: jay.krell at cornell.edu > > Why would we do this, and not signalling, as in POSIX implementation? > > Why try to make something synchronous when we have working asynchronous > implementation? > > Why would it be better than sync one? > > On 19 Jan 2014, at 13:00, Jay K > > wrote: > > There is a list of threads maintained by the m3 runtime. > Not by enumerating OS threads. > The thread structure contains in it perhaps a boolean or enum as to > suspend requested, suspending, suspended, resuming, running, etc. > From jay.krell at cornell.edu Mon Jan 20 00:13:24 2014 From: jay.krell at cornell.edu (Jay K) Date: Sun, 19 Jan 2014 23:13:24 +0000 Subject: [M3devel] Win32 SuspendThread? In-Reply-To: <20140119200548.C18011A208F@async.async.caltech.edu> References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL> <9EC3F463-F248-4C79-AF3F-130D2252B8B0@gmail.com>, <20140119200548.C18011A208F@async.async.caltech.edu> Message-ID: "not worried" for native windows. Still worried about wow64. The answer for native is in Rotor/sscli and I have a program to show it working/not-working. Specifically SuspendThread needs to be followed by GetThreadContext to ensure the suspend has completed. If we don't do that, we must. If we already do, we are ok. I'll look later. wow64 is still a problem. I'm still looking into it and thinking about it. But we do have native AMD64_NT so just saying wow64 doesn't work might be the answer. I know people will protest that it works, but doesn't always. I haven't figured out what rotor/sscli does. Java uses cooperative suspend, which sidesteps this stuff. ?- Jay ---------------------------------------- > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Win32 SuspendThread? > Date: Sun, 19 Jan 2014 12:05:48 -0800 > From: mika at async.caltech.edu > > "less worried" = "I think it works"? > > I think the only threading that works in CM3 is user threading... > > I tried to run CM3-compiled binaries through "valgrind" but got an > error that CM3 is using the same signal that valgrind does internally > (on AMD64_LINUX at least). I think this is easy to change in m3core > (yes?) but haven't gotten around to it... > > valgrind has a threading checker called "helgrind"... > > Mika > > Jay writes: >> >>--Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A >>Content-Type: text/plain; >> charset=utf-8 >>Content-Transfer-Encoding: quoted-printable >> >>For native I'm less worried now. For wow64 I'm still worried & hope to follo= >>w up further. >> >> - Jay >> >>On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" wrote:= >> >> >>> Jay: >>> =20 >>> I=E2=80=99m very concerned about the threading not working properly on bot= >>h 32-bit and 64-bit Windows. >>> =20 >>> The Thread Test program crashes for me on both platforms. >>> =20 >>> I haven=E2=80=99t tried your new test program yet. >>> =20 >>> --Randy >>> =20 >>> From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K >>> Sent: Saturday, January 18, 2014 1:23 AM >>> To: m3devel >>> Subject: EXT:[M3devel] Win32 SuspendThread? >>> =20 >>> This program also doesn't behave as expected, native, nothing to do with w= >>ow64.=20 >>> Anyone else please confirm:=20 >>> 1) my expectations -- it should never print anything >>> 2) their importance -- garbage collector depends on it =20 >>> 3) their not being met -- stuff gets printed=20 >>> This is in the CVS repository, scratch/wow64stack/sync2.cpp=20 >>> I am following up further.=20 >>> Maybe we should get cooperative suspend really going?=20 >>> Thank you. >>> - Jay=20 >>> =20 >>> #include >>> #include >>> volatile long value; >>> unsigned long __stdcall Thread(PVOID parameter) >>> { >>> while (1) >>> InterlockedIncrement(&value); >>> return 0; >>> } >>> int __cdecl main() >>> { >>> HANDLE thread =3D CreateThread(0, 0, Thread, 0, 0, 0); >>> UINT i =3D 0; >>> while (1) >>> { >>> i +=3D 1; >>> if (SuspendThread(thread) =3D=3D (DWORD)-1) >>> { >>> printf("suspend failed %X\n", GetLastError()); >>> Sleep(1); >>> continue; >>> } >>> volatile long a =3D value;=20 >>> volatile long b =3D value; >>> if (a !=3D b) >>> { >>> printf("%d %d %d %d\n", i, a, b, b - a); >>> } >>> ResumeThread(thread); >>> } >>> } >> >>--Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A >>Content-Type: text/html; >> charset=utf-8 >>Content-Transfer-Encoding: quoted-printable >> >>>utf-8">
For native I'm less worried now. For w= >>ow64 I'm still worried & hope to follow up further.

 - Jay>div>

On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" <>mailto:rcolebur at SCIRES.COM">rcolebur at SCIRES.COM> wrote:

= >>
>> >>= >> >> >> >>
I understand your reluctance.>>

>=3B Is the goal just to get FreeBSD to have a binary with the l= >>atest
>=3B sources?


Yes=2C and to make sure the build proce= >>ss works for you.


>=3B Shouldn't FreeBSD use the C backend too= >>?


Every target should. I haven't convinced everyone yet.
Ther= >>e is less resistance for new targets=2C like AMD64_NT and ARM_LINUX=2C whic= >>h are working=2C using the C backend.
There are downsides to the C backe= >>nd:
 =3B - it is slower to compiler =3B
 =3B - debugging= >> is degraded on systems that have m3gdb =3B
 =3B(debugging is m= >>uch better on systems w/o m3gdb)


And there are advantages not y= >>et implemented=2C in particular=2C efficient exception handling by generati= >>ng C++.


>=3B >=3B gcc/amd64 works for the same
>=3B >= >>=3B for Linux=2C FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.
>= >>=3B
>=3B I assume you mean after the system understands this target. = >> There will
>=3B be a lot of files with __FreeBSD__ that need to be ch= >>anged to
>=3B __FreeBSD__ || __DragonFly__ as well. In any case=2C I'= >>ll have to
>=3B regenerate a bunch of patches from my current work.>>

Your patches should apply fairly cleanly to any new checkout.
I= >> am willing to through them and apply them to CVS if you want -- depends on= >> if you are ok with my stealing your credit=2C vs. you want to commit them = >>yourselves.
I already skimmed them and they all look easy.
DragonflyB= >>SD is basically FreeBSD by another name.
Yes=2C a ton of work has been d= >>one on the kernel=2C file systems=2C maybe ports.
But the ABI is almost = >>the same -- heck=2C for the most part=2C Linux=3D=3DNetBSD=3D=3DFreeBSD=3D= >>=3DOpenBSD.
Sure=2C they might be implemented differently=2C but they ar= >>e highly source compatible and every highly object code compatible.

= >>
>=3B I must be misunderstanding what you mean by "upgrade". It's the= >>
>=3B cross-compiler that's complaining=2C and it was built by the lat= >>est patches.

Agreed=2C we are both missing something simple.
Usua= >>lly this is the error you get when you don't upgrade Target.i3/Target.m3.>r>If you want to press on with the current approach and ignore my advise=2C= >> it should be possible to step through all this.
To start=2C break on Ta= >>rget__Init and see if it returns true or false.


>=3B staticall= >>y while a bug in the cm3 sources caused those libraries to link
>=3B d= >>ynamically (which is why release binaries don't run on current FreeBSD)
= >>

Ugh. I saw something about that in your diffs. Definitely we intend= >> to link statically.
I actually went to the trouble of removing use of g= >>mp/mpfr/mpc in current source.
They are used for actually very little an= >>d aren't worth it.


 =3B- Jay


>=3B Date: Su= >>n=2C 19 Jan 2014 13:38:32 +0100
>=3B From: adacore at marino.st
>=3B= >> To: jay.krell at cornell.edu
>=3B CC: m3devel at elegosoft.com
>=3B Su= >>bject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY)
&g= >>t=3B
>=3B On 1/19/2014 13:17=2C Jay K wrote:
>=3B >=3B Hey..I = >>like that that does capture some of My Way.
>=3B >=3B Here is my in= >>dependent off the cuff write up though:
>=3B >=3B I know the ports = >>system is nice.
>=3B >=3B I know we really need to be in it for man= >>y operating systems.
>=3B >=3B I suspect your problem is that you d= >>idn't "upgrade" to apply you
>=3B >=3B changes at the right time.>>>=3B
>=3B I must be misunderstanding what you mean by "upgrade". = >>It's the
>=3B cross-compiler that's complaining=2C and it was built by= >> the latest patches.
>=3B
>=3B
>=3B >=3B Do any of thes= >>e work for you:
>=3B >=3B https://modula3.elegosoft.com/cm3/snaps/s= >>napshot-index.html
>=3B
>=3B I'm not clear to which set you are = >>referring.
>=3B There's no need for a binary set=2C modula3 now builds= >> on FreeBSD 9 and
>=3B FreeBSD 10 (http://www.freshports.org/lang/modu= >>la3)=2C which is also
>=3B superior to the release binaries because th= >>ey link gmp and mpfr
>=3B statically while a bug in the cm3 sources ca= >>used those libraries to link
>=3B dynamically (which is why release bi= >>naries don't run on current FreeBSD)
>=3B
>=3B Then there are th= >>e source distributions=2C but they are from 2012 so they
>=3B aren't e= >>xactly fresh either.
>=3B
>=3B
>=3B >=3B Anyway=2C plea= >>se try this:
>=3B >=3B Ignore DragonFly at first.
>=3B >= >>=3B Go to a system with a working cm3. Such as FreeBSD. Linux. Almost
= >>>=3B >=3B anything.
>=3B >=3B Such as from ports.
>=3B = >>>=3B Make a backup=2C e.g. of /usr/local/cm3.
>=3B >=3B My in= >>structions are unfortunately destructive of the existing install.
>= >>=3B >=3B Checkout the full current CVS repository.
>=3B >=3B = >>In that repository=2C run scripts/python/upgrade.py.
>=3B >=3B If= >> that doesn't work=2C stop=2C and we'll help.
>=3B
>=3B Is the = >>goal just to get FreeBSD to have a binary with the latest
>=3B sources= >>? If so=2C I'd just modify the port to do that now that we have
>=3B = >>working binary bootstraps. The main reason I'm reluctant is that it
>= >>=3B took over a week to make the cm3 port=2C I've never had so much difficu= >>lty
>=3B before. (I've also spent another week on the DragonFly port)= >>. So I'm
>=3B "resisting" because of how much work it already took. = >>That's why I was
>=3B trying to port DragonFly=2C THEN update to lates= >>t rather than update to
>=3B latest and then port DragonFly. (in othe= >>r words=2C going forward seems a
>=3B short path than backtracking)>>>=3B
>=3B
>=3B
>=3B >=3B If that does work=2C appl= >>y your diffs. And upgrade.py again. You can say
>=3B >=3B "upgrade.p= >>y skipgcc" here.
>=3B >=3B Then boot1.py c amd64_dragonflybsd <= >>br>>=3B >=3B Make sure you say "c" and the target "amd64_dragonflyb= >>sd"=2C anywhere
>=3B >=3B on the command line.
>=3B >=3B = >> Ok=2C "c" is actually up to you. Since nobody messes with ABI much=2C
&= >>gt=3B >=3B gcc/amd64 works for the same
>=3B >=3B for Linux=2C= >> FreeBSD=2C OpenBSD=2C NetBSD=2C and Dragonfly.
>=3B
>=3B I assu= >>me you mean after the system understands this target. There will
>=3B= >> be a lot of files with __FreeBSD__ that need to be changed to
>=3B __= >>FreeBSD__ || __DragonFly__ as well. In any case=2C I'll have to
>=3B = >>regenerate a bunch of patches from my current work.
>=3B
>=3B >r>>=3B >=3B If this works=2C it will give you=2C roughly amd64_dragon= >>flybsd*.tar.gz
>=3B >=3B Copy that to your Dragonfly system=2C ex= >>tract it=2C cd into it=2C make.
>=3B >=3B If that works=2C you ha= >>ve a native cm3.
>=3B >=3B Run it. It should say "can't find cm3.= >>cfg".
>=3B >=3B Now on Dragonfly=2C mkdir -p /usr/local/cm3/bin = >>
>=3B >=3B cp cm3 /usr/local/cm3/bin
>=3B >=3B export P= >>ATH=3D/usr/local/cm3/bin:$PATH
>=3B >=3B full CVS checkout
&g= >>t=3B >=3B cd scripts/python =3B ./boot2.sh
>=3B >=3B
>= >>=3B >=3B You will need to have made small changes to pylib.py for this = >>to work.
>=3B >=3B (And your config file should say to use the C b= >>ackend=2C like Darwin and
>=3B >=3B ARM_LINUX do.)
>=3B
>= >>=3B
>=3B Well=2C I'm thrilled if I don't have to patch gcc again=2C t= >>hat takes a long
>=3B time. Shouldn't FreeBSD use the C backend too?<= >>br>>=3B
>=3B John
>>= >> >>--_f56ed036-02b3-494e-bb04-f7c58d07fa28_-- From mika at async.caltech.edu Mon Jan 20 00:39:38 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sun, 19 Jan 2014 15:39:38 -0800 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: References: <52DBA2EF.4020101@marino.st>, , <52DBC081.1070401@marino.st> , <52DBC748.5090308@marino.st> , <20140119201308.0A4B41A208F@async.async.caltech.edu> Message-ID: <20140119233938.9C5A31A208F@async.async.caltech.edu> Jay K writes: >The only problem I remember is an alignment matter using msvcrt.dll on AMD6= >4_NT.=0A= >I'll look into it=2C and the old emails.=0A= >I've used the C backend on many platforms now.=0A= This stuff doesn't happen with the native backend: > 2. I can't get anything with Trestle working. I'm running in a VNC > session on a remote machine. Not sure this has anything to do with the > compiler or even the ARM. Has anyone seen it before (on any system)? > > (gdb) cont > Continuing. > [New Thread 0x42276430 (LWP 24451)] > [Thread 0x42276430 (LWP 24451) exited] > [New Thread 0x424c6430 (LWP 24454)] > > > *** > *** runtime error: > *** Exception "ZChildVBT.BadPercentage" not in RAISES list > *** file "../src/lego/ZChildVBT.m3", line 61 > *** > > > Program received signal SIGABRT, Aborted. > > backtrace: > > (gdb) where > #0 ZChildVBT__Init (z_L_120=0x422bbf70 "H\357K", ch_L_121=0x422bdb84 "\260\326K", h_L_122=0, v_L_123=1.75, loc_L_124=4 '\004', type_L_125=1 '\001', shaper_ L_126=0x41a32b54, open_L_127=0 '\000') at ../src/lego/ZCh > ildVBT.m3:61 > #1 0x40637518 in FormsVBT__pZChild (cl_L_1042=0x42280444, list_L_1043=0x424c54c4, s_L_1044=0x424c55dc) at ../src/FormsVBT.m3:2593 > #2 0x40609f30 in FormsVBT__Item (cl_L_301=0x42280444, exp_L_302=0x41a45994 "", state_L_303=0x424c55dc) at ../src/FormsVBT.m3:250 > #3 0x40648b3c in FormsVBT__AddChildren (cl_L_1229=0x42280444, v_L_1230=0x422b0634 " \365K", list_L_1231=0x41a48504, state_L_1232=0x424c55dc) at ../src/FormsVBT.m3:3671 > #4 0x40634138 in FormsVBT__pZSplit (cl_L_999=0x42280444, list_L_1000=0x424c56bc, s_L_1001=0x424c57c4) at ../src/FormsVBT.m3:2454 > #5 0x40609f30 in FormsVBT__Item (cl_L_301=0x42280444, exp_L_302=0x41a40ec0 "", state_L_303=0x424c57c4) at ../src/FormsVBT.m3:250 > #6 0x4064838c in FormsVBT__OneChild (cl_L_1224=0x42280444, list_L_1225=0x0, state_L_1226=0x424c57c4) at ../src/FormsVBT.m3:3642 > #7 0x406129c0 in FormsVBT__pShape (cl_L_496=0x42280444, list_L_497=0x424c58a4, s_L_498=0x424c59a4) at ../src/FormsVBT.m3:948 > #8 0x40609f30 in FormsVBT__Item (cl_L_301=0x42280444, exp_L_302=0x41a402a4 "", state_L_303=0x424c59a4) at ../src/FormsVBT.m3:250 > #9 0x4064838c in FormsVBT__OneChild (cl_L_1224=0x42280444, list_L_1225=0x0, state_L_1226=0x424c59a4) at ../src/FormsVBT.m3:3642 > #10 0x4062ca10 in FormsVBT__pScale (cl_L_905=0x42280444, list_L_906=0x424c5a84, s_L_907=0x42280458) at ../src/FormsVBT.m3:2165 > #11 0x40609f30 in FormsVBT__Item (cl_L_301=0x42280444, exp_L_302=0x41a4006c "", state_L_303=0x42280458) at ../src/FormsVBT.m3:250 > #12 0x40606dec in FormsVBT__Apply (cl_L_293=0x42280444) at ../src/FormsVBT.m3:84 > #13 0x40d27bd4 in ThreadPThread__RunThread (me_L_231=0x4c5d78 "0SLB\330]L") at ../src/thread/PTHREAD/ThreadPThread.m3:449 > #14 0x40d27590 in ThreadPThread__ThreadBase (param_L_229=0x4c5d78 "0SLB\330]L") at ../src/thread/PTHREAD/ThreadPThread.m3:422 > #15 0x41850bfc in start_thread () from /lib/arm-linux-gnueabihf/libpthread.so.0 > #16 0x41933758 in ?? () from /lib/arm-linux-gnueabihf/libc.so.6 > #17 0x41933758 in ?? () from /lib/arm-linux-gnueabihf/libc.so.6 > Backtrace stopped: previous frame identical to this frame (corrupt stack?) > (gdb) cont > Continuing. > From jay.krell at cornell.edu Mon Jan 20 01:06:27 2014 From: jay.krell at cornell.edu (Jay) Date: Sun, 19 Jan 2014 16:06:27 -0800 Subject: [M3devel] Win32 SuspendThread? In-Reply-To: <20140119200548.C18011A208F@async.async.caltech.edu> References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252412D@ATLEX04-SRV.SCIRES.LOCAL> <9EC3F463-F248-4C79-AF3F-130D2252B8B0@gmail.com> <20140119200548.C18011A208F@async.async.caltech.edu> Message-ID: Cooperative suspend will also fix this signal allocation vs. valgrind problem. Can also likely pick a different one... - Jay On Jan 19, 2014, at 12:05 PM, wrote: > "less worried" = "I think it works"? > > I think the only threading that works in CM3 is user threading... > > I tried to run CM3-compiled binaries through "valgrind" but got an > error that CM3 is using the same signal that valgrind does internally > (on AMD64_LINUX at least). I think this is easy to change in m3core > (yes?) but haven't gotten around to it... > > valgrind has a threading checker called "helgrind"... > > Mika > > Jay writes: >> >> --Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A >> Content-Type: text/plain; >> charset=utf-8 >> Content-Transfer-Encoding: quoted-printable >> >> For native I'm less worried now. For wow64 I'm still worried & hope to follo= >> w up further. >> >> - Jay >> >> On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" wrote:= >> >> >>> Jay: >>> =20 >>> I=E2=80=99m very concerned about the threading not working properly on bot= >> h 32-bit and 64-bit Windows. >>> =20 >>> The Thread Test program crashes for me on both platforms. >>> =20 >>> I haven=E2=80=99t tried your new test program yet. >>> =20 >>> --Randy >>> =20 >>> From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K >>> Sent: Saturday, January 18, 2014 1:23 AM >>> To: m3devel >>> Subject: EXT:[M3devel] Win32 SuspendThread? >>> =20 >>> This program also doesn't behave as expected, native, nothing to do with w= >> ow64.=20 >>> Anyone else please confirm:=20 >>> 1) my expectations -- it should never print anything >>> 2) their importance -- garbage collector depends on it =20 >>> 3) their not being met -- stuff gets printed=20 >>> This is in the CVS repository, scratch/wow64stack/sync2.cpp=20 >>> I am following up further.=20 >>> Maybe we should get cooperative suspend really going?=20 >>> Thank you. >>> - Jay=20 >>> =20 >>> #include >>> #include >>> volatile long value; >>> unsigned long __stdcall Thread(PVOID parameter) >>> { >>> while (1) >>> InterlockedIncrement(&value); >>> return 0; >>> } >>> int __cdecl main() >>> { >>> HANDLE thread =3D CreateThread(0, 0, Thread, 0, 0, 0); >>> UINT i =3D 0; >>> while (1) >>> { >>> i +=3D 1; >>> if (SuspendThread(thread) =3D=3D (DWORD)-1) >>> { >>> printf("suspend failed %X\n", GetLastError()); >>> Sleep(1); >>> continue; >>> } >>> volatile long a =3D value;=20 >>> volatile long b =3D value; >>> if (a !=3D b) >>> { >>> printf("%d %d %d %d\n", i, a, b, b - a); >>> } >>> ResumeThread(thread); >>> } >>> } >> >> --Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A >> Content-Type: text/html; >> charset=utf-8 >> Content-Transfer-Encoding: quoted-printable >> >> > utf-8">
For native I'm less worried now. For w= >> ow64 I'm still worried & hope to follow up further.

 - Jay> div>

On Jan 17, 2014, at 11:11 PM, "Coleburn, Randy" <> mailto:rcolebur at SCIRES.COM">rcolebur at SCIRES.COM> wrote:

= >>
>> >> = >> >> >> >> >> >>
>>

> ibri","sans-serif";color:#1F497D">Jay:

>>

> ibri","sans-serif";color:#1F497D"> >

> ibri","sans-serif";color:#1F497D">I=E2=80=99m very concerned a= >> bout the threading not working properly on both 32-bit and 64-bit Windows.> :p>

>>

> ibri","sans-serif";color:#1F497D"> >

> ibri","sans-serif";color:#1F497D">The Thread Test program cra= >> shes for me on both platforms.

>>

> ibri","sans-serif";color:#1F497D"> >

> ibri","sans-serif";color:#1F497D">I haven=E2=80=99t tried you= >> r new test program yet.

>>

> ibri","sans-serif";color:#1F497D"> >

> ibri","sans-serif";color:#1F497D">--Randy> p> >>

> ibri","sans-serif";color:#1F497D"> >

>>
> in 0in"> >>

> e:10.0pt;font-family:"Tahoma","sans-serif"">From:= >> > s-serif""> jayk123 at hotmail.com> a> [mailto:jayk123 at hotmail.com] >> On Behalf Of Jay K
>> Sent: Saturday, January 18, 2014 1:23 AM
>> To: m3devel
>> Subject: EXT:[M3devel] Win32 SuspendThread?

>>
>>
>>

 

>>
>>

> in-bottom:12.0pt;margin-left:.5in"> >>  = >> ;This program also doesn't behave as expected, native, nothing to do with wo= >> w64. >>
>>  Anyone else please confirm:
>>    1) my expectations -- it should never print anything
>>    2) their importance  -- garbage collector depends on it&nb= >> sp;
>>    3) their not being met -- stuff gets printed
>>  This is in the CVS repository, scratch/wow64stack/sync2.cpp
>>  I am following up further.
>>  Maybe we should get cooperative suspend really going?
>> Thank you.
>>  - Jay
>>  
>> #include <stdio.h>
>> #include <windows.h>
>> volatile long value;
>> unsigned long __stdcall Thread(PVOID parameter)
>> {
>>   while (1)
>>    InterlockedIncrement(&value);
>>   return 0;
>> }
>> int __cdecl main()
>> {
>>   HANDLE thread =3D CreateThread(0, 0, Thread, 0, 0, 0);
>>   UINT i =3D 0;
>>   while (1)
>>   {
>>     i +=3D 1;
>>     if (SuspendThread(thread) =3D=3D (DWORD)-1)
>>     {
>>       printf("suspend failed %X\n", GetLastError())= >> ;
>>       Sleep(1);
>>       continue;
>>     }
>>     volatile long a =3D value;
>>     volatile long b =3D value;
>>     if (a !=3D b)
>>     {
>>      printf("%d %d %d %d\n", i, a, b, b - a);
>>     }
>>     ResumeThread(thread);
>>   }
>> }

>>
>>
>> >> >>
= >> >> --Apple-Mail-79A8190D-B4F0-41F1-AA37-366ED17CF55A-- From rodney_bates at lcwb.coop Mon Jan 20 17:00:13 2014 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 20 Jan 2014 10:00:13 -0600 Subject: [M3devel] narrow still failing.. In-Reply-To: <20140119220123.0D1971A208F@async.async.caltech.edu> References: <20140119213649.403AE1A208F@async.async.caltech.edu> <20140119220123.0D1971A208F@async.async.caltech.edu> Message-ID: <52DD480D.3000101@lcwb.coop> What does this do? PROCEDURE Z(VAR x : REFANY) = VAR b: BDD.T; BEGIN b := x; x := BDD.Format(b) END Z; The common property of the failing cases seems to be the necessity of a runtime narrow check when passing x to BDD.Format. So far, I haven't been able to reproduce the failure on AMD64_LINUX, with even more irrelevant (one would think) stuff pared out. I moved T, Root, and Format into main, removed all but one fields of T and Root, replaced BDD.New by NEW(Root) and removed Symtab and most of the body from Format. Looking at your code, I agree it should work. On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote: > As in the ISTYPE/TYPECASE example, the following works: > > > PROCEDURE Z(VAR x : REFANY) = > BEGIN > TYPECASE x OF > BDD.T(b) => x := BDD.Format(b) > END; > END Z; > > (244)rover:~/refany/src>../FreeBSD4/refany > a > > > This does NOT work: > > PROCEDURE Z(VAR x : REFANY) = > BEGIN > TYPECASE x OF > BDD.T(b) => x := BDD.Format(x) > END; > END Z; > > > (242)rover:~/refany/src>../FreeBSD4/refany > > > *** > *** runtime error: > *** NARROW failed > *** file "/big/home/mika/refany/src/Main.m3", line 22 > *** > > use option @M3stackdump to get a stack trace > Abort > > > Same result PM3 / CM3. > > Mika > From rodney_bates at lcwb.coop Mon Jan 20 18:02:40 2014 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 20 Jan 2014 11:02:40 -0600 Subject: [M3devel] narrow still failing.. In-Reply-To: <20140119220123.0D1971A208F@async.async.caltech.edu> References: <20140119213649.403AE1A208F@async.async.caltech.edu> <20140119220123.0D1971A208F@async.async.caltech.edu> Message-ID: <52DD56B0.1050007@lcwb.coop> Lots more experiments, still no reproduction of the problem on AMD64_LINUX. The 5 cases in the program below all work. But earlier versions of this test program, with bugs, (failure to repeat x:=y before every Z* call) suggested a hypothesis. For x := BDD.Format(x) Perhaps the compiler does something to the LHS x (zeros it out?) before narrowing and passing the x that is an actual parameter to Format. If so, it is an implementation bug, as the language says (2.3.1): (for v := e) "The order of evaluation of v and e is undefined, but e will be evaluated before v is updated." _____________________________________________________________________________________ MODULE Main; TYPE T = OBJECT l : T; END; TYPE Root = T OBJECT id : CARDINAL; END; PROCEDURE Format(x : T) : TEXT = BEGIN RETURN "Format" END Format; PROCEDURE P() : REFANY = BEGIN VAR x, y : REFANY; BEGIN y := NEW ( Root ); x := y; Z1(x); x := y; Z2(x); x := y; Z3(x); x := y; Z4(x); x := y; Z5(x); RETURN x END END P; PROCEDURE Z1(VAR x : REFANY) = BEGIN TYPECASE x OF T (b) => x := Format(b) END; END Z1; PROCEDURE Z2(VAR x : REFANY) = VAR b : T; BEGIN b := x; x := Format(b) END Z2; PROCEDURE Z3(VAR x : REFANY) = BEGIN x := Format(x) END Z3; PROCEDURE Z4(VAR x : REFANY) = BEGIN TYPECASE x OF T => x := Format (x) END; END Z4; PROCEDURE Z5(VAR x : REFANY) = BEGIN IF ISTYPE (x, T) THEN x := Format (x) END; END Z5; BEGIN EVAL P() END Main. ______________________________________________________________________________________ On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote: > As in the ISTYPE/TYPECASE example, the following works: > > > PROCEDURE Z(VAR x : REFANY) = > BEGIN > TYPECASE x OF > BDD.T(b) => x := BDD.Format(b) > END; > END Z; > > (244)rover:~/refany/src>../FreeBSD4/refany > a > > > This does NOT work: > > PROCEDURE Z(VAR x : REFANY) = > BEGIN > TYPECASE x OF > BDD.T(b) => x := BDD.Format(x) > END; > END Z; > > > (242)rover:~/refany/src>../FreeBSD4/refany > > > *** > *** runtime error: > *** NARROW failed > *** file "/big/home/mika/refany/src/Main.m3", line 22 > *** > > use option @M3stackdump to get a stack trace > Abort > > > Same result PM3 / CM3. > > Mika > From mika at async.caltech.edu Tue Jan 21 05:18:26 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Mon, 20 Jan 2014 20:18:26 -0800 Subject: [M3devel] narrow still failing.. In-Reply-To: <52DD480D.3000101@lcwb.coop> References: <20140119213649.403AE1A208F@async.async.caltech.edu> <20140119220123.0D1971A208F@async.async.caltech.edu> <52DD480D.3000101@lcwb.coop> Message-ID: <20140121041826.4D95D1A2080@async.async.caltech.edu> Were you able to get it to fail on any of your computers? Your Z fails for me---CM3 and PM3 both. (579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany *** *** runtime error: *** An explicit or implicit NARROW() operation failed. *** file "../src/Main.m3", line 30 *** Abort (580)truffles:~/bsd/refany/src>cat Mai Main.m3~ Main.m3 (580)truffles:~/bsd/refany/src>cat -n Main.m3 1 MODULE Main; 2 IMPORT BDD, IO; 3 4 PROCEDURE P() : REFANY = 5 BEGIN 6 VAR x : REFANY; 7 BEGIN 8 x := BDD.New("a"); 9 Z(x); 10 RETURN x 11 END 12 END P; 13 14 PROCEDURE Q() = 15 BEGIN 16 IO.Put(P() & "\n") 17 END Q; 18 19 (* 20 PROCEDURE Z(VAR x : REFANY) = 21 BEGIN 22 TYPECASE x OF 23 BDD.T(b) => x := BDD.Format(b) 24 END; 25 END Z; 26 *) 27 PROCEDURE Z(VAR x : REFANY) = 28 VAR b: BDD.T; 29 BEGIN 30 b := x; 31 x := BDD.Format(b) 32 END Z; 33 34 35 36 BEGIN 37 Q() 38 END Main. (581)truffles:~/bsd/refany/src> "Rodney M. Bates" writes: >What does this do? > >PROCEDURE Z(VAR x : REFANY) = > VAR b: BDD.T; > BEGIN > b := x; > x := BDD.Format(b) > END Z; > >The common property of the failing cases seems to be the necessity >of a runtime narrow check when passing x to BDD.Format. > >So far, I haven't been able to reproduce the failure on AMD64_LINUX, >with even more irrelevant (one would think) stuff pared out. I moved >T, Root, and Format into main, removed all but one fields of T and Root, >replaced BDD.New by NEW(Root) and removed Symtab and most of the body >from Format. > >Looking at your code, I agree it should work. > > >On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote: >> As in the ISTYPE/TYPECASE example, the following works: >> >> >> PROCEDURE Z(VAR x : REFANY) = >> BEGIN >> TYPECASE x OF >> BDD.T(b) => x := BDD.Format(b) >> END; >> END Z; >> >> (244)rover:~/refany/src>../FreeBSD4/refany >> a >> >> >> This does NOT work: >> >> PROCEDURE Z(VAR x : REFANY) = >> BEGIN >> TYPECASE x OF >> BDD.T(b) => x := BDD.Format(x) >> END; >> END Z; >> >> >> (242)rover:~/refany/src>../FreeBSD4/refany >> >> >> *** >> *** runtime error: >> *** NARROW failed >> *** file "/big/home/mika/refany/src/Main.m3", line 22 >> *** >> >> use option @M3stackdump to get a stack trace >> Abort >> >> >> Same result PM3 / CM3. >> >> Mika >> From rodney_bates at lcwb.coop Tue Jan 21 17:34:57 2014 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 21 Jan 2014 10:34:57 -0600 Subject: [M3devel] narrow still failing.. In-Reply-To: <20140121041826.4D95D1A2080@async.async.caltech.edu> References: <20140119213649.403AE1A208F@async.async.caltech.edu> <20140119220123.0D1971A208F@async.async.caltech.edu> <52DD480D.3000101@lcwb.coop> <20140121041826.4D95D1A2080@async.async.caltech.edu> Message-ID: <52DEA1B1.2070808@lcwb.coop> On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote: > Were you able to get it to fail on any of your computers? No, the only times I got narrow failures were due to errors in my trial cases. I have only tried AMD64_LINUX, but should be able to get this onto a LINUXLIBC6 system. > > Your Z fails for me---CM3 and PM3 both. > > (579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany > > > *** > *** runtime error: > *** An explicit or implicit NARROW() operation failed. > *** file "../src/Main.m3", line 30 > *** Well that torpedos both my theories. But it is hard to imagine that bad code generated for an implicit narrow on an assignment could ever have gone unnoticed so long. I thought it might be more believable if it only happened when narrowing an actual parameter before passing it. > > Abort > (580)truffles:~/bsd/refany/src>cat Mai > Main.m3~ Main.m3 > (580)truffles:~/bsd/refany/src>cat -n Main.m3 > 1 MODULE Main; > 2 IMPORT BDD, IO; > 3 > 4 PROCEDURE P() : REFANY = > 5 BEGIN > 6 VAR x : REFANY; > 7 BEGIN > 8 x := BDD.New("a"); > 9 Z(x); > 10 RETURN x > 11 END > 12 END P; > 13 > 14 PROCEDURE Q() = > 15 BEGIN > 16 IO.Put(P() & "\n") > 17 END Q; > 18 > 19 (* > 20 PROCEDURE Z(VAR x : REFANY) = > 21 BEGIN > 22 TYPECASE x OF > 23 BDD.T(b) => x := BDD.Format(b) > 24 END; > 25 END Z; > 26 *) > 27 PROCEDURE Z(VAR x : REFANY) = > 28 VAR b: BDD.T; > 29 BEGIN > 30 b := x; > 31 x := BDD.Format(b) > 32 END Z; > 33 > 34 > 35 > 36 BEGIN > 37 Q() > 38 END Main. > (581)truffles:~/bsd/refany/src> > "Rodney M. Bates" writes: >> What does this do? >> >> PROCEDURE Z(VAR x : REFANY) = >> VAR b: BDD.T; >> BEGIN >> b := x; >> x := BDD.Format(b) >> END Z; >> >> The common property of the failing cases seems to be the necessity >> of a runtime narrow check when passing x to BDD.Format. >> >> So far, I haven't been able to reproduce the failure on AMD64_LINUX, >> with even more irrelevant (one would think) stuff pared out. I moved >> T, Root, and Format into main, removed all but one fields of T and Root, >> replaced BDD.New by NEW(Root) and removed Symtab and most of the body >>from Format. >> >> Looking at your code, I agree it should work. >> >> >> On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote: >>> As in the ISTYPE/TYPECASE example, the following works: >>> >>> >>> PROCEDURE Z(VAR x : REFANY) = >>> BEGIN >>> TYPECASE x OF >>> BDD.T(b) => x := BDD.Format(b) >>> END; >>> END Z; >>> >>> (244)rover:~/refany/src>../FreeBSD4/refany >>> a >>> >>> >>> This does NOT work: >>> >>> PROCEDURE Z(VAR x : REFANY) = >>> BEGIN >>> TYPECASE x OF >>> BDD.T(b) => x := BDD.Format(x) >>> END; >>> END Z; >>> >>> >>> (242)rover:~/refany/src>../FreeBSD4/refany >>> >>> >>> *** >>> *** runtime error: >>> *** NARROW failed >>> *** file "/big/home/mika/refany/src/Main.m3", line 22 >>> *** >>> >>> use option @M3stackdump to get a stack trace >>> Abort >>> >>> >>> Same result PM3 / CM3. >>> >>> Mika >>> > From mika at async.caltech.edu Tue Jan 21 18:11:20 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Tue, 21 Jan 2014 09:11:20 -0800 Subject: [M3devel] narrow still failing.. In-Reply-To: <52DEA1B1.2070808@lcwb.coop> References: <20140119213649.403AE1A208F@async.async.caltech.edu> <20140119220123.0D1971A208F@async.async.caltech.edu> <52DD480D.3000101@lcwb.coop> <20140121041826.4D95D1A2080@async.async.caltech.edu> <52DEA1B1.2070808@lcwb.coop> Message-ID: <20140121171120.F21FF1A2080@async.async.caltech.edu> Did you try using my bdd.tgz ? I am getting this error on: ancient PM3 on FreeBSD4 head CM3 on AMD64_LINUX few months old CM3 on ARM_LINUX (Raspberry Pi) I doubt you will get it on LINUXLIBC6 if you don't get it on the others. I already tried rearranging the files (the implementation and interfaces are split in a funny way across BDD.i3, BDDImpl.i3, BDD.m3, BDDImpl.m3 right now). With no effect. I don't know what it is about my code that's tripping up the compiler. It's also a "decent" BDD package so someone (me I guess, but anyone else who's interested is welcome to) might want to add it to CM3... :-) Kind of a pain if you can't print out the debugging info though! (Actually the workarounds are OK.) Mika "Rodney M. Bates" writes: > > >On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote: >> Were you able to get it to fail on any of your computers? > >No, the only times I got narrow failures were due to errors >in my trial cases. > >I have only tried AMD64_LINUX, but should be able to get this onto a >LINUXLIBC6 system. > >> >> Your Z fails for me---CM3 and PM3 both. >> >> (579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany >> >> >> *** >> *** runtime error: >> *** An explicit or implicit NARROW() operation failed. >> *** file "../src/Main.m3", line 30 >> *** > >Well that torpedos both my theories. > >But it is hard to imagine that bad code generated for an implicit narrow >on an assignment could ever have gone unnoticed so long. I thought it >might be more believable if it only happened when narrowing an actual >parameter before passing it. > >> >> Abort >> (580)truffles:~/bsd/refany/src>cat Mai >> Main.m3~ Main.m3 >> (580)truffles:~/bsd/refany/src>cat -n Main.m3 >> 1 MODULE Main; >> 2 IMPORT BDD, IO; >> 3 >> 4 PROCEDURE P() : REFANY = >> 5 BEGIN >> 6 VAR x : REFANY; >> 7 BEGIN >> 8 x := BDD.New("a"); >> 9 Z(x); >> 10 RETURN x >> 11 END >> 12 END P; >> 13 >> 14 PROCEDURE Q() = >> 15 BEGIN >> 16 IO.Put(P() & "\n") >> 17 END Q; >> 18 >> 19 (* >> 20 PROCEDURE Z(VAR x : REFANY) = >> 21 BEGIN >> 22 TYPECASE x OF >> 23 BDD.T(b) => x := BDD.Format(b) >> 24 END; >> 25 END Z; >> 26 *) >> 27 PROCEDURE Z(VAR x : REFANY) = >> 28 VAR b: BDD.T; >> 29 BEGIN >> 30 b := x; >> 31 x := BDD.Format(b) >> 32 END Z; >> 33 >> 34 >> 35 >> 36 BEGIN >> 37 Q() >> 38 END Main. >> (581)truffles:~/bsd/refany/src> >> "Rodney M. Bates" writes: >>> What does this do? >>> >>> PROCEDURE Z(VAR x : REFANY) = >>> VAR b: BDD.T; >>> BEGIN >>> b := x; >>> x := BDD.Format(b) >>> END Z; >>> >>> The common property of the failing cases seems to be the necessity >>> of a runtime narrow check when passing x to BDD.Format. >>> >>> So far, I haven't been able to reproduce the failure on AMD64_LINUX, >>> with even more irrelevant (one would think) stuff pared out. I moved >>> T, Root, and Format into main, removed all but one fields of T and Root, >>> replaced BDD.New by NEW(Root) and removed Symtab and most of the body >>>from Format. >>> >>> Looking at your code, I agree it should work. >>> >>> >>> On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote: >>>> As in the ISTYPE/TYPECASE example, the following works: >>>> >>>> >>>> PROCEDURE Z(VAR x : REFANY) = >>>> BEGIN >>>> TYPECASE x OF >>>> BDD.T(b) => x := BDD.Format(b) >>>> END; >>>> END Z; >>>> >>>> (244)rover:~/refany/src>../FreeBSD4/refany >>>> a >>>> >>>> >>>> This does NOT work: >>>> >>>> PROCEDURE Z(VAR x : REFANY) = >>>> BEGIN >>>> TYPECASE x OF >>>> BDD.T(b) => x := BDD.Format(x) >>>> END; >>>> END Z; >>>> >>>> >>>> (242)rover:~/refany/src>../FreeBSD4/refany >>>> >>>> >>>> *** >>>> *** runtime error: >>>> *** NARROW failed >>>> *** file "/big/home/mika/refany/src/Main.m3", line 22 >>>> *** >>>> >>>> use option @M3stackdump to get a stack trace >>>> Abort >>>> >>>> >>>> Same result PM3 / CM3. >>>> >>>> Mika >>>> >> From rodney_bates at lcwb.coop Tue Jan 21 18:23:41 2014 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 21 Jan 2014 11:23:41 -0600 Subject: [M3devel] narrow still failing.. In-Reply-To: <20140121041826.4D95D1A2080@async.async.caltech.edu> References: <20140119213649.403AE1A208F@async.async.caltech.edu> <20140119220123.0D1971A208F@async.async.caltech.edu> <52DD480D.3000101@lcwb.coop> <20140121041826.4D95D1A2080@async.async.caltech.edu> Message-ID: <52DEAD1D.8010602@lcwb.coop> On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote: > Were you able to get it to fail on any of your computers? > My test case with 5 versions of Z, and your complete one that you posted (bdd.tgz) both silently complete with no error messages on LINUXLIBC6, using the head compiler and libraries. bdd.tgz also works fine on AMD64_LINUX. On yours, I built and shipped in bdd/src, then built in bdd/test/src and ran LINUXLIBC6/bddtest, which I presume was the right procedure. How about compiling a couple of failing Z versions on a machine where they fail, with -keep, then posting the .mc, .ms, and .mo files and corresponding source (only for the Zs is probably enough). >>> Mika >>> > From rodney_bates at lcwb.coop Tue Jan 21 19:02:09 2014 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 21 Jan 2014 12:02:09 -0600 Subject: [M3devel] narrow still failing.. In-Reply-To: <20140121171120.F21FF1A2080@async.async.caltech.edu> References: <20140119213649.403AE1A208F@async.async.caltech.edu> <20140119220123.0D1971A208F@async.async.caltech.edu> <52DD480D.3000101@lcwb.coop> <20140121041826.4D95D1A2080@async.async.caltech.edu> <52DEA1B1.2070808@lcwb.coop> <20140121171120.F21FF1A2080@async.async.caltech.edu> Message-ID: <52DEB621.7080706@lcwb.coop> On 01/21/2014 11:11 AM, mika at async.caltech.edu wrote: > Did you try using my bdd.tgz ? > > I am getting this error on: > > ancient PM3 on FreeBSD4 > > head CM3 on AMD64_LINUX Hmm. I can't get any errors compiling bdd and bddtest on my AMD64_LINUX system with a freshly updated head. Hacking around, I tried to use the binaries in bdd.tgz, but my .M3SHIP wants a libbdd.so.5, not there. Some of the *.[im]o files are slightly different in size when I recompiled, but I think there is an explanation for that. I can't build bddtest with the release compiler, because it needs caltech_parser, and that won't build in the release. This combination of [non]failures seems pretty strange. I am also wondering about a problem with type checks in the runtime. > > few months old CM3 on ARM_LINUX (Raspberry Pi) > > I doubt you will get it on LINUXLIBC6 if you don't get it on the others. > > I already tried rearranging the files (the implementation and interfaces > are split in a funny way across BDD.i3, BDDImpl.i3, BDD.m3, BDDImpl.m3 > right now). With no effect. I don't know what it is about my code that's > tripping up the compiler. > > It's also a "decent" BDD package so someone (me I guess, but anyone > else who's interested is welcome to) might want to add it to CM3... :-) > Kind of a pain if you can't print out the debugging info though! > (Actually the workarounds are OK.) > > Mika > > "Rodney M. Bates" writes: >> >> >> On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote: >>> Were you able to get it to fail on any of your computers? >> >> No, the only times I got narrow failures were due to errors >> in my trial cases. >> >> I have only tried AMD64_LINUX, but should be able to get this onto a >> LINUXLIBC6 system. >> >>> >>> Your Z fails for me---CM3 and PM3 both. >>> >>> (579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany >>> >>> >>> *** >>> *** runtime error: >>> *** An explicit or implicit NARROW() operation failed. >>> *** file "../src/Main.m3", line 30 >>> *** >> >> Well that torpedos both my theories. >> >> But it is hard to imagine that bad code generated for an implicit narrow >> on an assignment could ever have gone unnoticed so long. I thought it >> might be more believable if it only happened when narrowing an actual >> parameter before passing it. >> >>> >>> Abort >>> (580)truffles:~/bsd/refany/src>cat Mai >>> Main.m3~ Main.m3 >>> (580)truffles:~/bsd/refany/src>cat -n Main.m3 >>> 1 MODULE Main; >>> 2 IMPORT BDD, IO; >>> 3 >>> 4 PROCEDURE P() : REFANY = >>> 5 BEGIN >>> 6 VAR x : REFANY; >>> 7 BEGIN >>> 8 x := BDD.New("a"); >>> 9 Z(x); >>> 10 RETURN x >>> 11 END >>> 12 END P; >>> 13 >>> 14 PROCEDURE Q() = >>> 15 BEGIN >>> 16 IO.Put(P() & "\n") >>> 17 END Q; >>> 18 >>> 19 (* >>> 20 PROCEDURE Z(VAR x : REFANY) = >>> 21 BEGIN >>> 22 TYPECASE x OF >>> 23 BDD.T(b) => x := BDD.Format(b) >>> 24 END; >>> 25 END Z; >>> 26 *) >>> 27 PROCEDURE Z(VAR x : REFANY) = >>> 28 VAR b: BDD.T; >>> 29 BEGIN >>> 30 b := x; >>> 31 x := BDD.Format(b) >>> 32 END Z; >>> 33 >>> 34 >>> 35 >>> 36 BEGIN >>> 37 Q() >>> 38 END Main. >>> (581)truffles:~/bsd/refany/src> >>> "Rodney M. Bates" writes: >>>> What does this do? >>>> >>>> PROCEDURE Z(VAR x : REFANY) = >>>> VAR b: BDD.T; >>>> BEGIN >>>> b := x; >>>> x := BDD.Format(b) >>>> END Z; >>>> >>>> The common property of the failing cases seems to be the necessity >>>> of a runtime narrow check when passing x to BDD.Format. >>>> >>>> So far, I haven't been able to reproduce the failure on AMD64_LINUX, >>>> with even more irrelevant (one would think) stuff pared out. I moved >>>> T, Root, and Format into main, removed all but one fields of T and Root, >>>> replaced BDD.New by NEW(Root) and removed Symtab and most of the body >>> >from Format. >>>> >>>> Looking at your code, I agree it should work. >>>> >>>> >>>> On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote: >>>>> As in the ISTYPE/TYPECASE example, the following works: >>>>> >>>>> >>>>> PROCEDURE Z(VAR x : REFANY) = >>>>> BEGIN >>>>> TYPECASE x OF >>>>> BDD.T(b) => x := BDD.Format(b) >>>>> END; >>>>> END Z; >>>>> >>>>> (244)rover:~/refany/src>../FreeBSD4/refany >>>>> a >>>>> >>>>> >>>>> This does NOT work: >>>>> >>>>> PROCEDURE Z(VAR x : REFANY) = >>>>> BEGIN >>>>> TYPECASE x OF >>>>> BDD.T(b) => x := BDD.Format(x) >>>>> END; >>>>> END Z; >>>>> >>>>> >>>>> (242)rover:~/refany/src>../FreeBSD4/refany >>>>> >>>>> >>>>> *** >>>>> *** runtime error: >>>>> *** NARROW failed >>>>> *** file "/big/home/mika/refany/src/Main.m3", line 22 >>>>> *** >>>>> >>>>> use option @M3stackdump to get a stack trace >>>>> Abort >>>>> >>>>> >>>>> Same result PM3 / CM3. >>>>> >>>>> Mika >>>>> >>> > From mika at async.caltech.edu Tue Jan 21 19:08:06 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Tue, 21 Jan 2014 10:08:06 -0800 Subject: [M3devel] narrow still failing.. In-Reply-To: <52DEB621.7080706@lcwb.coop> References: <20140119213649.403AE1A208F@async.async.caltech.edu> <20140119220123.0D1971A208F@async.async.caltech.edu> <52DD480D.3000101@lcwb.coop> <20140121041826.4D95D1A2080@async.async.caltech.edu> <52DEA1B1.2070808@lcwb.coop> <20140121171120.F21FF1A2080@async.async.caltech.edu> <52DEB621.7080706@lcwb.coop> Message-ID: <20140121180806.D75A61A208F@async.async.caltech.edu> The only thing bddtest uses from caltech_parser is Debug.Out. You can just take out those calls or replace them with IO.Put. But again bddtest is not the program that fails... it's the "refany" program (now in a sibling directory to bddtest) Mika "Rodney M. Bates" writes: > > >On 01/21/2014 11:11 AM, mika at async.caltech.edu wrote: >> Did you try using my bdd.tgz ? >> >> I am getting this error on: >> >> ancient PM3 on FreeBSD4 >> >> head CM3 on AMD64_LINUX > >Hmm. I can't get any errors compiling bdd and bddtest on my AMD64_LINUX system with >a freshly updated head. Hacking around, I tried to use the binaries in bdd.tgz, >but my .M3SHIP wants a libbdd.so.5, not there. Some of the *.[im]o files are >slightly different in size when I recompiled, but I think there is an explanation >for that. > >I can't build bddtest with the release compiler, because it needs caltech_parser, >and that won't build in the release. This combination of [non]failures seems >pretty strange. I am also wondering about a problem with type checks in the >runtime. > > >> >> few months old CM3 on ARM_LINUX (Raspberry Pi) >> >> I doubt you will get it on LINUXLIBC6 if you don't get it on the others. >> >> I already tried rearranging the files (the implementation and interfaces >> are split in a funny way across BDD.i3, BDDImpl.i3, BDD.m3, BDDImpl.m3 >> right now). With no effect. I don't know what it is about my code that's >> tripping up the compiler. >> >> It's also a "decent" BDD package so someone (me I guess, but anyone >> else who's interested is welcome to) might want to add it to CM3... :-) >> Kind of a pain if you can't print out the debugging info though! >> (Actually the workarounds are OK.) >> >> Mika >> >> "Rodney M. Bates" writes: >>> >>> >>> On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote: >>>> Were you able to get it to fail on any of your computers? >>> >>> No, the only times I got narrow failures were due to errors >>> in my trial cases. >>> >>> I have only tried AMD64_LINUX, but should be able to get this onto a >>> LINUXLIBC6 system. >>> >>>> >>>> Your Z fails for me---CM3 and PM3 both. >>>> >>>> (579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany >>>> >>>> >>>> *** >>>> *** runtime error: >>>> *** An explicit or implicit NARROW() operation failed. >>>> *** file "../src/Main.m3", line 30 >>>> *** >>> >>> Well that torpedos both my theories. >>> >>> But it is hard to imagine that bad code generated for an implicit narrow >>> on an assignment could ever have gone unnoticed so long. I thought it >>> might be more believable if it only happened when narrowing an actual >>> parameter before passing it. >>> >>>> >>>> Abort >>>> (580)truffles:~/bsd/refany/src>cat Mai >>>> Main.m3~ Main.m3 >>>> (580)truffles:~/bsd/refany/src>cat -n Main.m3 >>>> 1 MODULE Main; >>>> 2 IMPORT BDD, IO; >>>> 3 >>>> 4 PROCEDURE P() : REFANY = >>>> 5 BEGIN >>>> 6 VAR x : REFANY; >>>> 7 BEGIN >>>> 8 x := BDD.New("a"); >>>> 9 Z(x); >>>> 10 RETURN x >>>> 11 END >>>> 12 END P; >>>> 13 >>>> 14 PROCEDURE Q() = >>>> 15 BEGIN >>>> 16 IO.Put(P() & "\n") >>>> 17 END Q; >>>> 18 >>>> 19 (* >>>> 20 PROCEDURE Z(VAR x : REFANY) = >>>> 21 BEGIN >>>> 22 TYPECASE x OF >>>> 23 BDD.T(b) => x := BDD.Format(b) >>>> 24 END; >>>> 25 END Z; >>>> 26 *) >>>> 27 PROCEDURE Z(VAR x : REFANY) = >>>> 28 VAR b: BDD.T; >>>> 29 BEGIN >>>> 30 b := x; >>>> 31 x := BDD.Format(b) >>>> 32 END Z; >>>> 33 >>>> 34 >>>> 35 >>>> 36 BEGIN >>>> 37 Q() >>>> 38 END Main. >>>> (581)truffles:~/bsd/refany/src> >>>> "Rodney M. Bates" writes: >>>>> What does this do? >>>>> >>>>> PROCEDURE Z(VAR x : REFANY) = >>>>> VAR b: BDD.T; >>>>> BEGIN >>>>> b := x; >>>>> x := BDD.Format(b) >>>>> END Z; >>>>> >>>>> The common property of the failing cases seems to be the necessity >>>>> of a runtime narrow check when passing x to BDD.Format. >>>>> >>>>> So far, I haven't been able to reproduce the failure on AMD64_LINUX, >>>>> with even more irrelevant (one would think) stuff pared out. I moved >>>>> T, Root, and Format into main, removed all but one fields of T and Root, >>>>> replaced BDD.New by NEW(Root) and removed Symtab and most of the body >>>> >from Format. >>>>> >>>>> Looking at your code, I agree it should work. >>>>> >>>>> >>>>> On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote: >>>>>> As in the ISTYPE/TYPECASE example, the following works: >>>>>> >>>>>> >>>>>> PROCEDURE Z(VAR x : REFANY) = >>>>>> BEGIN >>>>>> TYPECASE x OF >>>>>> BDD.T(b) => x := BDD.Format(b) >>>>>> END; >>>>>> END Z; >>>>>> >>>>>> (244)rover:~/refany/src>../FreeBSD4/refany >>>>>> a >>>>>> >>>>>> >>>>>> This does NOT work: >>>>>> >>>>>> PROCEDURE Z(VAR x : REFANY) = >>>>>> BEGIN >>>>>> TYPECASE x OF >>>>>> BDD.T(b) => x := BDD.Format(x) >>>>>> END; >>>>>> END Z; >>>>>> >>>>>> >>>>>> (242)rover:~/refany/src>../FreeBSD4/refany >>>>>> >>>>>> >>>>>> *** >>>>>> *** runtime error: >>>>>> *** NARROW failed >>>>>> *** file "/big/home/mika/refany/src/Main.m3", line 22 >>>>>> *** >>>>>> >>>>>> use option @M3stackdump to get a stack trace >>>>>> Abort >>>>>> >>>>>> >>>>>> Same result PM3 / CM3. >>>>>> >>>>>> Mika >>>>>> >>>> >> From mika at async.caltech.edu Tue Jan 21 20:23:28 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Tue, 21 Jan 2014 11:23:28 -0800 Subject: [M3devel] narrow still failing.. In-Reply-To: <07CB74B6-895A-4053-9999-1D1A46C404D0@purdue.edu> References: <20140119213649.403AE1A208F@async.async.caltech.edu> <20140119220123.0D1971A208F@async.async.caltech.edu> <52DD480D.3000101@lcwb.coop> <20140121041826.4D95D1A2080@async.async.caltech.edu> <52DEA1B1.2070808@lcwb.coop> <20140121171120.F21FF1A2080@async.async.caltech.edu> <52DEB621.7080706@lcwb.coop> <20140121180806.D75A61A208F@async.async.caltech.edu> <07CB74B6-895A-4053-9999-1D1A46C404D0@purdue.edu> Message-ID: <20140121192328.92DDB1A208F@async.async.caltech.edu> How do you get them into text form? They look binary... I copied the whole directory bdd (and all subdirs) to http://rover.gcapltd.com/~mika/bdd/ Antony Hosking writes: >Please post .io / .mo in text form. > >Sent from my iPad > >> On Jan 21, 2014, at 1:08 PM, mika at async.caltech.edu wrote: >>=20 >> The only thing bddtest uses from caltech_parser is Debug.Out. You can jus= >t take out those calls or replace them with IO.Put. >>=20 >> But again bddtest is not the program that fails... it's the "refany" progr= >am (now in a sibling directory to bddtest) >>=20 >> Mika >>=20 >> "Rodney M. Bates" writes: >>>=20 >>>=20 >>>> On 01/21/2014 11:11 AM, mika at async.caltech.edu wrote: >>>> Did you try using my bdd.tgz ? >>>>=20 >>>> I am getting this error on: >>>>=20 >>>> ancient PM3 on FreeBSD4 >>>>=20 >>>> head CM3 on AMD64_LINUX >>>=20 >>> Hmm. I can't get any errors compiling bdd and bddtest on my AMD64_LINUX s= >ystem with >>> a freshly updated head. Hacking around, I tried to use the binaries in b= >dd.tgz, >>> but my .M3SHIP wants a libbdd.so.5, not there. Some of the *.[im]o files= > are >>> slightly different in size when I recompiled, but I think there is an exp= >lanation >>> for that. >>>=20 >>> I can't build bddtest with the release compiler, because it needs caltech= >_parser, >>> and that won't build in the release. This combination of [non]failures s= >eems >>> pretty strange. I am also wondering about a problem with type checks in t= >he >>> runtime. >>>=20 >>>=20 >>>>=20 >>>> few months old CM3 on ARM_LINUX (Raspberry Pi) >>>>=20 >>>> I doubt you will get it on LINUXLIBC6 if you don't get it on the others.= > >>>>=20 >>>> I already tried rearranging the files (the implementation and interfaces= > >>>> are split in a funny way across BDD.i3, BDDImpl.i3, BDD.m3, BDDImpl.m3 >>>> right now). With no effect. I don't know what it is about my code that= >'s >>>> tripping up the compiler. >>>>=20 >>>> It's also a "decent" BDD package so someone (me I guess, but anyone >>>> else who's interested is welcome to) might want to add it to CM3... :-) >>>> Kind of a pain if you can't print out the debugging info though! >>>> (Actually the workarounds are OK.) >>>>=20 >>>> Mika >>>>=20 >>>> "Rodney M. Bates" writes: >>>>>=20 >>>>>=20 >>>>>> On 01/20/2014 10:18 PM, mika at async.caltech.edu wrote: >>>>>> Were you able to get it to fail on any of your computers? >>>>>=20 >>>>> No, the only times I got narrow failures were due to errors >>>>> in my trial cases. >>>>>=20 >>>>> I have only tried AMD64_LINUX, but should be able to get this onto a >>>>> LINUXLIBC6 system. >>>>>=20 >>>>>>=20 >>>>>> Your Z fails for me---CM3 and PM3 both. >>>>>>=20 >>>>>> (579)truffles:~/bsd/refany/src>../AMD64_LINUX/refany >>>>>>=20 >>>>>>=20 >>>>>> *** >>>>>> *** runtime error: >>>>>> *** An explicit or implicit NARROW() operation failed. >>>>>> *** file "../src/Main.m3", line 30 >>>>>> *** >>>>>=20 >>>>> Well that torpedos both my theories. >>>>>=20 >>>>> But it is hard to imagine that bad code generated for an implicit narro= >w >>>>> on an assignment could ever have gone unnoticed so long. I thought it >>>>> might be more believable if it only happened when narrowing an actual >>>>> parameter before passing it. >>>>>=20 >>>>>>=20 >>>>>> Abort >>>>>> (580)truffles:~/bsd/refany/src>cat Mai >>>>>> Main.m3~ Main.m3 >>>>>> (580)truffles:~/bsd/refany/src>cat -n Main.m3 >>>>>> 1 MODULE Main; >>>>>> 2 IMPORT BDD, IO; >>>>>> 3 >>>>>> 4 PROCEDURE P() : REFANY =3D >>>>>> 5 BEGIN >>>>>> 6 VAR x : REFANY; >>>>>> 7 BEGIN >>>>>> 8 x :=3D BDD.New("a"); >>>>>> 9 Z(x); >>>>>> 10 RETURN x >>>>>> 11 END >>>>>> 12 END P; >>>>>> 13 >>>>>> 14 PROCEDURE Q() =3D >>>>>> 15 BEGIN >>>>>> 16 IO.Put(P() & "\n") >>>>>> 17 END Q; >>>>>> 18 >>>>>> 19 (* >>>>>> 20 PROCEDURE Z(VAR x : REFANY) =3D >>>>>> 21 BEGIN >>>>>> 22 TYPECASE x OF >>>>>> 23 BDD.T(b) =3D> x :=3D BDD.Format(b) >>>>>> 24 END; >>>>>> 25 END Z; >>>>>> 26 *) >>>>>> 27 PROCEDURE Z(VAR x : REFANY) =3D >>>>>> 28 VAR b: BDD.T; >>>>>> 29 BEGIN >>>>>> 30 b :=3D x; >>>>>> 31 x :=3D BDD.Format(b) >>>>>> 32 END Z; >>>>>> 33 >>>>>> 34 >>>>>> 35 >>>>>> 36 BEGIN >>>>>> 37 Q() >>>>>> 38 END Main. >>>>>> (581)truffles:~/bsd/refany/src> >>>>>> "Rodney M. Bates" writes: >>>>>>> What does this do? >>>>>>>=20 >>>>>>> PROCEDURE Z(VAR x : REFANY) =3D >>>>>>> VAR b: BDD.T; >>>>>>> BEGIN >>>>>>> b :=3D x; >>>>>>> x :=3D BDD.Format(b) >>>>>>> END Z; >>>>>>>=20 >>>>>>> The common property of the failing cases seems to be the necessity >>>>>>> of a runtime narrow check when passing x to BDD.Format. >>>>>>>=20 >>>>>>> So far, I haven't been able to reproduce the failure on AMD64_LINUX, >>>>>>> with even more irrelevant (one would think) stuff pared out. I moved= > >>>>>>> T, Root, and Format into main, removed all but one fields of T and Ro= >ot, >>>>>>> replaced BDD.New by NEW(Root) and removed Symtab and most of the body= > >>>>>>> from Format. >>>>>>>=20 >>>>>>> Looking at your code, I agree it should work. >>>>>>>=20 >>>>>>>=20 >>>>>>>> On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote: >>>>>>>> As in the ISTYPE/TYPECASE example, the following works: >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> PROCEDURE Z(VAR x : REFANY) =3D >>>>>>>> BEGIN >>>>>>>> TYPECASE x OF >>>>>>>> BDD.T(b) =3D> x :=3D BDD.Format(b) >>>>>>>> END; >>>>>>>> END Z; >>>>>>>>=20 >>>>>>>> (244)rover:~/refany/src>../FreeBSD4/refany >>>>>>>> a >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> This does NOT work: >>>>>>>>=20 >>>>>>>> PROCEDURE Z(VAR x : REFANY) =3D >>>>>>>> BEGIN >>>>>>>> TYPECASE x OF >>>>>>>> BDD.T(b) =3D> x :=3D BDD.Format(x) >>>>>>>> END; >>>>>>>> END Z; >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> (242)rover:~/refany/src>../FreeBSD4/refany >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> *** >>>>>>>> *** runtime error: >>>>>>>> *** NARROW failed >>>>>>>> *** file "/big/home/mika/refany/src/Main.m3", line 22 >>>>>>>> *** >>>>>>>>=20 >>>>>>>> use option @M3stackdump to get a stack trace >>>>>>>> Abort >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> Same result PM3 / CM3. >>>>>>>>=20 >>>>>>>> Mika >>>>=20 From rodney_bates at lcwb.coop Wed Jan 22 20:46:55 2014 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 22 Jan 2014 13:46:55 -0600 Subject: [M3devel] narrow still failing.. In-Reply-To: <20140119220123.0D1971A208F@async.async.caltech.edu> References: <20140119213649.403AE1A208F@async.async.caltech.edu> <20140119220123.0D1971A208F@async.async.caltech.edu> Message-ID: <52E0202F.3020307@lcwb.coop> Some more info: The front end is generating incorrect narrow code, only where the type T is an unrevealed <: REFANY. When T and Root are known, it calls RTHooks.CheckIsType for ISTYPE, NARROW, and implicit narrow in an assignment. When opaque, for explicit and implicit narrow, it simply compares the typecode of x to a single typecode (no doubt for T) and demands they be equal. In this case, the actual allocated type of x is Root, a proper subtype of T. For ISTYPE, it does call CheckIsType even when the types are opaque. The fact that it happens only when the types are opaque lends credibility to the fact that this bug has gone unnoticed at least as far back as PM3. On 01/19/2014 04:01 PM, mika at async.caltech.edu wrote: > As in the ISTYPE/TYPECASE example, the following works: > > > PROCEDURE Z(VAR x : REFANY) = > BEGIN > TYPECASE x OF > BDD.T(b) => x := BDD.Format(b) > END; > END Z; > > (244)rover:~/refany/src>../FreeBSD4/refany > a > > > This does NOT work: > > PROCEDURE Z(VAR x : REFANY) = > BEGIN > TYPECASE x OF > BDD.T(b) => x := BDD.Format(x) > END; > END Z; > > > (242)rover:~/refany/src>../FreeBSD4/refany > > > *** > *** runtime error: > *** NARROW failed > *** file "/big/home/mika/refany/src/Main.m3", line 22 > *** > > use option @M3stackdump to get a stack trace > Abort > > > Same result PM3 / CM3. > > Mika > From mika at async.caltech.edu Wed Jan 22 21:47:49 2014 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Wed, 22 Jan 2014 12:47:49 -0800 Subject: [M3devel] narrow still failing.. In-Reply-To: <52E0202F.3020307@lcwb.coop> References: <20140119213649.403AE1A208F@async.async.caltech.edu> <20140119220123.0D1971A208F@async.async.caltech.edu> <52E0202F.3020307@lcwb.coop> Message-ID: <20140122204749.C724E1A208F@async.async.caltech.edu> Aha so declaring the type as <: ROOT will probably work as a workaround? Thank you for looking into this. "Rodney M. Bates" writes: >Some more info: > >The front end is generating incorrect narrow code, only >where the type T is an unrevealed <: REFANY. When T and Root >are known, it calls RTHooks.CheckIsType for ISTYPE, NARROW, >and implicit narrow in an assignment. > >When opaque, for explicit and implicit narrow, it simply compares >the typecode of x to a single typecode (no doubt for T) and demands >they be equal. In this case, the actual allocated type of x is Root, >a proper subtype of T. > >For ISTYPE, it does call CheckIsType even when the types are opaque. > >The fact that it happens only when the types are opaque lends >credibility to the fact that this bug has gone unnoticed at least >as far back as PM3. From jay.krell at cornell.edu Thu Jan 23 08:51:58 2014 From: jay.krell at cornell.edu (Jay K) Date: Thu, 23 Jan 2014 07:51:58 +0000 Subject: [M3devel] proposal for getting wow64 context/stack reliably? Message-ID: There is a behavior/bug in wow64. Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc. SuspendThread / GetThreadContext work like this: 32bit processes consist almost entirely of 32bit code. There is a small amount of 64bit code. If you suspend while running 32bit code, GetThreadContext works. if you suspend while running 64bit code, GetThreadContext usually but not always works. 64bit code is run en route to syscalls. For example you call: 1 kernel32!Sleep 2 it calls 32bit NtDelayExecution 3 that calls wow64NtDelayExecutation (via a cross segment "far" jmp or call) 4 which calls native NtDelayExecuation In between 2 and 3, within 64bit code, the 32bit context is saved. You can step through it very easily in a debugger. Really. Where GetThreadContext knows where to get it. The problem is that saving context is not atomic. You can suspend while saving context. What to do? scratch/wow64stack contains a program that detects the bug. I believe it is the basis of a workaround for the bug. Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, not only functions that use exception handling, but also functions that call "extern" functions call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection use that, if it isn't zero. Normally it will be zero. Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile. That is why it isn't a set/set-zero pattern (like in the test program). Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. Rejected counter proposal: Don't suspend/gc when running a syscall. No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that. Possible augmentation: if running native, short circuit most of this Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, until/unless there is another 32bit platform runable on some similar 64bit platforms. Performance impact: hypothetically large but probably not noticable. Furthe refinements: It isn't extern/native code per se, it is syscalls. We could further augment pragmas to discern them. We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... Agreed? I'll make the compiler change? Oh, also, not just stack pointer, but other registers, at least non-volatiles? Eventually cooperative suspend will cause this to fall away as a problem.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcolebur at SCIRES.COM Thu Jan 23 12:59:34 2014 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Thu, 23 Jan 2014 11:59:34 +0000 Subject: [M3devel] EXT: proposal for getting wow64 context/stack reliably? In-Reply-To: References: Message-ID: Jay, If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows? --Randy Sent from my iPhone On Jan 23, 2014, at 2:58 AM, "Jay K" > wrote: There is a behavior/bug in wow64. Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc. SuspendThread / GetThreadContext work like this: 32bit processes consist almost entirely of 32bit code. There is a small amount of 64bit code. If you suspend while running 32bit code, GetThreadContext works. if you suspend while running 64bit code, GetThreadContext usually but not always works. 64bit code is run en route to syscalls. For example you call: 1 kernel32!Sleep 2 it calls 32bit NtDelayExecution 3 that calls wow64NtDelayExecutation (via a cross segment "far" jmp or call) 4 which calls native NtDelayExecuation In between 2 and 3, within 64bit code, the 32bit context is saved. You can step through it very easily in a debugger. Really. Where GetThreadContext knows where to get it. The problem is that saving context is not atomic. You can suspend while saving context. What to do? scratch/wow64stack contains a program that detects the bug. I believe it is the basis of a workaround for the bug. Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, not only functions that use exception handling, but also functions that call "extern" functions call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection use that, if it isn't zero. Normally it will be zero. Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile. That is why it isn't a set/set-zero pattern (like in the test program). Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. Rejected counter proposal: Don't suspend/gc when running a syscall. No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that. Possible augmentation: if running native, short circuit most of this Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, until/unless there is another 32bit platform runable on some similar 64bit platforms. Performance impact: hypothetically large but probably not noticable. Furthe refinements: It isn't extern/native code per se, it is syscalls. We could further augment pragmas to discern them. We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... Agreed? I'll make the compiler change? Oh, also, not just stack pointer, but other registers, at least non-volatiles? Eventually cooperative suspend will cause this to fall away as a problem.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Thu Jan 23 16:25:59 2014 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 23 Jan 2014 09:25:59 -0600 Subject: [M3devel] narrow still failing.. In-Reply-To: <20140122204749.C724E1A208F@async.async.caltech.edu> References: <20140119213649.403AE1A208F@async.async.caltech.edu> <20140119220123.0D1971A208F@async.async.caltech.edu> <52E0202F.3020307@lcwb.coop> <20140122204749.C724E1A208F@async.async.caltech.edu> Message-ID: <52E13487.9090504@lcwb.coop> On 01/22/2014 02:47 PM, mika at async.caltech.edu wrote: > Aha so declaring the type as <: ROOT will probably work as a workaround? > I'm not sure whether that would do it or not. In my pared-down case, I had moved BDD.T and BDD.Root into main, but left the REFANY variables. That made the failures go away. Until it is fixed, I think the best workaround is TYPECASE with a binding: TYPECASE x OF BDD.T(b) => Format(b), etc. This is known to be working now. You've probably thought about this, but this way requires only a single runtime type test. Either TYPECASE without the binding of b, or ISTYPE, will require an additional recheck of the same thing when the narrow, implicit or explicit, happens. I doubt even a pretty good optimizing compiler could remove this. For ISTYPE, it would have to know that the runtime routine (e.g. CheckIsType) was a pure function. Pretty unlikely. Even worse for TYPECASE, where it would have to know a lot about the relation between CheckIsType and ScanTypecase. > Thank you for looking into this. > > "Rodney M. Bates" writes: >> Some more info: >> >> The front end is generating incorrect narrow code, only >> where the type T is an unrevealed <: REFANY. When T and Root >> are known, it calls RTHooks.CheckIsType for ISTYPE, NARROW, >> and implicit narrow in an assignment. >> >> When opaque, for explicit and implicit narrow, it simply compares >> the typecode of x to a single typecode (no doubt for T) and demands >> they be equal. In this case, the actual allocated type of x is Root, >> a proper subtype of T. >> >> For ISTYPE, it does call CheckIsType even when the types are opaque. >> >> The fact that it happens only when the types are opaque lends >> credibility to the fact that this bug has gone unnoticed at least >> as far back as PM3. > From hosking at cs.purdue.edu Thu Jan 23 17:51:02 2014 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 23 Jan 2014 11:51:02 -0500 Subject: [M3devel] EXT: proposal for getting wow64 context/stack reliably? In-Reply-To: References: Message-ID: <4A941D84-FC5A-499A-8F7F-DE06CEFC0B37@cs.purdue.edu> Yes, agreed. We need to address the native thread problem on all platforms (both pthread and windows). However, this particular problem is pernicious if we need to save M3 stack state at all external calls. Are there any linker tricks for Windows one can use to wrap only system calls? I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped, This approach is used for all modern Java implementations that I am aware of. It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Mobile +1 765 427 5484 On Jan 23, 2014, at 6:59 AM, Coleburn, Randy wrote: > Jay, > > If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. > > Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. > > Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. > > Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows? > > --Randy > > Sent from my iPhone > > On Jan 23, 2014, at 2:58 AM, "Jay K" wrote: > >> There is a behavior/bug in wow64. >> Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc. >> >> >> >> SuspendThread / GetThreadContext work like this: >> >> >> 32bit processes consist almost entirely of 32bit code. >> There is a small amount of 64bit code. >> >> >> If you suspend while running 32bit code, GetThreadContext works. >> if you suspend while running 64bit code, GetThreadContext usually but not always works. >> >> >> 64bit code is run en route to syscalls. >> For example you call: >> 1 kernel32!Sleep >> 2 it calls 32bit NtDelayExecution >> 3 that calls wow64NtDelayExecutation (via a cross segment "far" jmp or call) >> 4 which calls native NtDelayExecuation >> >> >> In between 2 and 3, within 64bit code, the 32bit context is saved. >> You can step through it very easily in a debugger. Really. >> Where GetThreadContext knows where to get it. >> The problem is that saving context is not atomic. >> You can suspend while saving context. >> >> >> What to do? >> >> >> scratch/wow64stack contains a program that detects the bug. >> I believe it is the basis of a workaround for the bug. >> >> >> Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, >> not only functions that use exception handling, but also functions that call "extern" functions >> call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection >> use that, if it isn't zero. >> Normally it will be zero. >> Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile. >> That is why it isn't a set/set-zero pattern (like in the test program). >> >> >> Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. >> >> >> Rejected counter proposal: >> Don't suspend/gc when running a syscall. >> No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that. >> >> >> Possible augmentation: if running native, short circuit most of this >> >> >> Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate >> compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. >> >> >> AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, >> until/unless there is another 32bit platform runable on some similar 64bit platforms. >> >> >> Performance impact: hypothetically large but probably not noticable. >> >> >> Furthe refinements: It isn't extern/native code per se, it is syscalls. >> We could further augment pragmas to discern them. >> >> >> We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls >> to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. >> It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... >> >> >> >> Agreed? I'll make the compiler change? >> >> >> Oh, also, not just stack pointer, but other registers, at least non-volatiles? >> >> >> Eventually cooperative suspend will cause this to fall away as a problem.. >> >> >> - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jan 23 21:03:09 2014 From: jay.krell at cornell.edu (Jay K) Date: Thu, 23 Jan 2014 20:03:09 +0000 Subject: [M3devel] EXT: proposal for getting wow64 context/stack reliably? In-Reply-To: References: , , <4A941D84-FC5A-499A-8F7F-DE06CEFC0B37@cs.purdue.edu>, Message-ID: Syscalls are "doubly wrapped". Exports from ntdll.dll that start "Nt" or "Zw" are syscalls and nothing else. If you look at them, they all look special. But they expose a standard calling convention. Lesser known, there are also some syscalls in gdi32.dll and user32.dll. None of them are public interfaces. They are all reached through public interfaces typically in kernel32.dll, gdi32.dll, user32.dll. For example, kernel32!CreateFileW might eventually call ntdll!NtCreateFile, but not directly, and this can change. Unix is similar and different -- sometimes functions are just a syscall, sometimes they are wrappers that do other work. There are counter examples -- functions that just return global variables like GetTickCount/GetTickCount64/GetCommandLine/GetEnvironmentVariable without a syscall. Bottom line is though, one isn't really supposed to know where the syscalls are. - Jay From: jay.krell at cornell.edu To: hosking at cs.purdue.edu; rcolebur at scires.com CC: m3devel at elegosoft.com Subject: RE: [M3devel] EXT: proposal for getting wow64 context/stack reliably? Date: Thu, 23 Jan 2014 19:57:27 +0000 There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same. Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok. - Jay From: hosking at cs.purdue.edu Date: Thu, 23 Jan 2014 11:51:02 -0500 To: rcolebur at SCIRES.COM CC: m3devel at elegosoft.com; jay.krell at cornell.edu Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack reliably? Yes, agreed.We need to address the native thread problem on all platforms (both pthread and windows). However, this particular problem is pernicious if we need to save M3 stack state at all external calls. Are there any linker tricks for Windows one can use to wrap only system calls? I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped, This approach is used for all modern Java implementations that I am aware of. It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state. Antony Hosking | Associate Professor | Computer Science | Purdue University305 N. University Street | West Lafayette | IN 47907 | USAMobile +1 765 427 5484 On Jan 23, 2014, at 6:59 AM, Coleburn, Randy wrote:Jay, If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows? --Randy Sent from my iPhone On Jan 23, 2014, at 2:58 AM, "Jay K" wrote: There is a behavior/bug in wow64. Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc. SuspendThread / GetThreadContext work like this: 32bit processes consist almost entirely of 32bit code. There is a small amount of 64bit code. If you suspend while running 32bit code, GetThreadContext works. if you suspend while running 64bit code, GetThreadContext usually but not always works. 64bit code is run en route to syscalls. For example you call: 1 kernel32!Sleep 2 it calls 32bit NtDelayExecution 3 that calls wow64NtDelayExecutation (via a cross segment "far" jmp or call) 4 which calls native NtDelayExecuation In between 2 and 3, within 64bit code, the 32bit context is saved. You can step through it very easily in a debugger. Really. Where GetThreadContext knows where to get it. The problem is that saving context is not atomic. You can suspend while saving context. What to do? scratch/wow64stack contains a program that detects the bug. I believe it is the basis of a workaround for the bug. Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, not only functions that use exception handling, but also functions that call "extern" functions call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection use that, if it isn't zero. Normally it will be zero. Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile. That is why it isn't a set/set-zero pattern (like in the test program). Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. Rejected counter proposal: Don't suspend/gc when running a syscall. No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that. Possible augmentation: if running native, short circuit most of this Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, until/unless there is another 32bit platform runable on some similar 64bit platforms. Performance impact: hypothetically large but probably not noticable. Furthe refinements: It isn't extern/native code per se, it is syscalls. We could further augment pragmas to discern them. We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... Agreed? I'll make the compiler change? Oh, also, not just stack pointer, but other registers, at least non-volatiles? Eventually cooperative suspend will cause this to fall away as a problem.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jan 23 20:57:27 2014 From: jay.krell at cornell.edu (Jay K) Date: Thu, 23 Jan 2014 19:57:27 +0000 Subject: [M3devel] EXT: proposal for getting wow64 context/stack reliably? In-Reply-To: <4A941D84-FC5A-499A-8F7F-DE06CEFC0B37@cs.purdue.edu> References: , , <4A941D84-FC5A-499A-8F7F-DE06CEFC0B37@cs.purdue.edu> Message-ID: There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same. Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok. - Jay From: hosking at cs.purdue.edu Date: Thu, 23 Jan 2014 11:51:02 -0500 To: rcolebur at SCIRES.COM CC: m3devel at elegosoft.com; jay.krell at cornell.edu Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack reliably? Yes, agreed.We need to address the native thread problem on all platforms (both pthread and windows). However, this particular problem is pernicious if we need to save M3 stack state at all external calls. Are there any linker tricks for Windows one can use to wrap only system calls? I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped, This approach is used for all modern Java implementations that I am aware of. It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state. Antony Hosking | Associate Professor | Computer Science | Purdue University305 N. University Street | West Lafayette | IN 47907 | USAMobile +1 765 427 5484 On Jan 23, 2014, at 6:59 AM, Coleburn, Randy wrote:Jay, If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows? --Randy Sent from my iPhone On Jan 23, 2014, at 2:58 AM, "Jay K" wrote: There is a behavior/bug in wow64. Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc. SuspendThread / GetThreadContext work like this: 32bit processes consist almost entirely of 32bit code. There is a small amount of 64bit code. If you suspend while running 32bit code, GetThreadContext works. if you suspend while running 64bit code, GetThreadContext usually but not always works. 64bit code is run en route to syscalls. For example you call: 1 kernel32!Sleep 2 it calls 32bit NtDelayExecution 3 that calls wow64NtDelayExecutation (via a cross segment "far" jmp or call) 4 which calls native NtDelayExecuation In between 2 and 3, within 64bit code, the 32bit context is saved. You can step through it very easily in a debugger. Really. Where GetThreadContext knows where to get it. The problem is that saving context is not atomic. You can suspend while saving context. What to do? scratch/wow64stack contains a program that detects the bug. I believe it is the basis of a workaround for the bug. Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, not only functions that use exception handling, but also functions that call "extern" functions call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection use that, if it isn't zero. Normally it will be zero. Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile. That is why it isn't a set/set-zero pattern (like in the test program). Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. Rejected counter proposal: Don't suspend/gc when running a syscall. No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that. Possible augmentation: if running native, short circuit most of this Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, until/unless there is another 32bit platform runable on some similar 64bit platforms. Performance impact: hypothetically large but probably not noticable. Furthe refinements: It isn't extern/native code per se, it is syscalls. We could further augment pragmas to discern them. We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... Agreed? I'll make the compiler change? Oh, also, not just stack pointer, but other registers, at least non-volatiles? Eventually cooperative suspend will cause this to fall away as a problem.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcolebur at SCIRES.COM Thu Jan 23 23:23:43 2014 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Thu, 23 Jan 2014 22:23:43 +0000 Subject: [M3devel] proposal for getting wow64 context/stack reliably? Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL> Jay: When you say native 32/64-bit "should be ok", you really mean ok with respect to the SYSWOW64 problem you've identified. Both of these native implementations still have threading misbehaviors as identified by the thread test program. Correct? --Randy From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Thursday, January 23, 2014 2:57 PM To: Tony; Coleburn, Randy Cc: m3devel Subject: EXT:RE: [M3devel] EXT: proposal for getting wow64 context/stack reliably? There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same. Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok. - Jay ________________________________ From: hosking at cs.purdue.edu Date: Thu, 23 Jan 2014 11:51:02 -0500 To: rcolebur at SCIRES.COM CC: m3devel at elegosoft.com; jay.krell at cornell.edu Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack reliably? Yes, agreed. We need to address the native thread problem on all platforms (both pthread and windows). However, this particular problem is pernicious if we need to save M3 stack state at all external calls. Are there any linker tricks for Windows one can use to wrap only system calls? I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped, This approach is used for all modern Java implementations that I am aware of. It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Mobile +1 765 427 5484 On Jan 23, 2014, at 6:59 AM, Coleburn, Randy > wrote: Jay, If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows? --Randy Sent from my iPhone On Jan 23, 2014, at 2:58 AM, "Jay K" > wrote: There is a behavior/bug in wow64. Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc. SuspendThread / GetThreadContext work like this: 32bit processes consist almost entirely of 32bit code. There is a small amount of 64bit code. If you suspend while running 32bit code, GetThreadContext works. if you suspend while running 64bit code, GetThreadContext usually but not always works. 64bit code is run en route to syscalls. For example you call: 1 kernel32!Sleep 2 it calls 32bit NtDelayExecution 3 that calls wow64NtDelayExecutation (via a cross segment "far" jmp or call) 4 which calls native NtDelayExecuation In between 2 and 3, within 64bit code, the 32bit context is saved. You can step through it very easily in a debugger. Really. Where GetThreadContext knows where to get it. The problem is that saving context is not atomic. You can suspend while saving context. What to do? scratch/wow64stack contains a program that detects the bug. I believe it is the basis of a workaround for the bug. Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, not only functions that use exception handling, but also functions that call "extern" functions call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection use that, if it isn't zero. Normally it will be zero. Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile. That is why it isn't a set/set-zero pattern (like in the test program). Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. Rejected counter proposal: Don't suspend/gc when running a syscall. No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that. Possible augmentation: if running native, short circuit most of this Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, until/unless there is another 32bit platform runable on some similar 64bit platforms. Performance impact: hypothetically large but probably not noticable. Furthe refinements: It isn't extern/native code per se, it is syscalls. We could further augment pragmas to discern them. We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... Agreed? I'll make the compiler change? Oh, also, not just stack pointer, but other registers, at least non-volatiles? Eventually cooperative suspend will cause this to fall away as a problem.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Jan 24 00:09:50 2014 From: jay.krell at cornell.edu (Jay K) Date: Thu, 23 Jan 2014 23:09:50 +0000 Subject: [M3devel] proposal for getting wow64 context/stack reliably? In-Reply-To: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL> References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL> Message-ID: Yes. Agreed. They should be ok with respect to SuspendThread + GetThreadContext. The small C++ test program in scratch. I don't know about the "other" problem that affects even pthreads and I agree we should focus on that first, on pthreads, native I386_NT, and possibly AMD64_NT, but for now, at least briefly, ignore I386_NT on 64bit Windows. - Jay From: rcolebur at SCIRES.COM To: m3devel at elegosoft.com Date: Thu, 23 Jan 2014 22:23:43 +0000 Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably? Jay: When you say native 32/64-bit ?should be ok?, you really mean ok with respect to the SYSWOW64 problem you?ve identified. Both of these native implementations still have threading misbehaviors as identified by the thread test program. Correct? --Randy From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Thursday, January 23, 2014 2:57 PM To: Tony; Coleburn, Randy Cc: m3devel Subject: EXT:RE: [M3devel] EXT: proposal for getting wow64 context/stack reliably? There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same. Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok. - Jay From: hosking at cs.purdue.edu Date: Thu, 23 Jan 2014 11:51:02 -0500 To: rcolebur at SCIRES.COM CC: m3devel at elegosoft.com; jay.krell at cornell.edu Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack reliably? Yes, agreed. We need to address the native thread problem on all platforms (both pthread and windows). However, this particular problem is pernicious if we need to save M3 stack state at all external calls. Are there any linker tricks for Windows one can use to wrap only system calls? I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped, This approach is used for all modern Java implementations that I am aware of. It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Mobile +1 765 427 5484 On Jan 23, 2014, at 6:59 AM, Coleburn, Randy wrote: Jay, If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows? --Randy Sent from my iPhone On Jan 23, 2014, at 2:58 AM, "Jay K" wrote: There is a behavior/bug in wow64. Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc. SuspendThread / GetThreadContext work like this: 32bit processes consist almost entirely of 32bit code. There is a small amount of 64bit code. If you suspend while running 32bit code, GetThreadContext works. if you suspend while running 64bit code, GetThreadContext usually but not always works. 64bit code is run en route to syscalls. For example you call: 1 kernel32!Sleep 2 it calls 32bit NtDelayExecution 3 that calls wow64NtDelayExecutation (via a cross segment "far" jmp or call) 4 which calls native NtDelayExecuation In between 2 and 3, within 64bit code, the 32bit context is saved. You can step through it very easily in a debugger. Really. Where GetThreadContext knows where to get it. The problem is that saving context is not atomic. You can suspend while saving context. What to do? scratch/wow64stack contains a program that detects the bug. I believe it is the basis of a workaround for the bug. Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, not only functions that use exception handling, but also functions that call "extern" functions call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection use that, if it isn't zero. Normally it will be zero. Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile. That is why it isn't a set/set-zero pattern (like in the test program). Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. Rejected counter proposal: Don't suspend/gc when running a syscall. No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that. Possible augmentation: if running native, short circuit most of this Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, until/unless there is another 32bit platform runable on some similar 64bit platforms. Performance impact: hypothetically large but probably not noticable. Furthe refinements: It isn't extern/native code per se, it is syscalls. We could further augment pragmas to discern them. We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... Agreed? I'll make the compiler change? Oh, also, not just stack pointer, but other registers, at least non-volatiles? Eventually cooperative suspend will cause this to fall away as a problem.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Fri Jan 24 00:42:01 2014 From: adacore at marino.st (John Marino) Date: Fri, 24 Jan 2014 00:42:01 +0100 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: <52DBCDBA.8030809@marino.st> References: <52DBA2EF.4020101@marino.st>, , <52DBC081.1070401@marino.st> , <52DBC748.5090308@marino.st> <52DBCDBA.8030809@marino.st> Message-ID: <52E1A8C9.3010903@marino.st> On 1/19/2014 14:06, John Marino wrote: > On 1/19/2014 13:50, Jay K wrote: >> And there are advantages not yet implemented, in particular, efficient >> exception handling by generating C++. > > Like FreeBSD (and unlike NetBSD and OpenBSD), DragonFly has implemented > dl_iterate_phdr which gcc uses for efficient zero-cost exception > handling. That might come in handy here (I implemented it in > DragonFly's real-time linker, mainly for Ada/GNAT but C++ benefited.) > > >> Your patches should apply fairly cleanly to any new checkout. >> I am willing to through them and apply them to CVS if you want -- >> depends on if you are ok with my stealing your credit, vs. you want to >> commit them yourselves. > > I would love it if you did this - I'm happy with a mention in the commit > message. I just attached a tarball of all the patches I'm using right > now. Some of these changes are specifically for the cross compiler (I'm > think of the /scripts changes mostly) but most are valid assuming they > still apply to the head of the repo. Please take as much as you want > with my gratitude. > >> I already skimmed them and they all look easy. >> DragonflyBSD is basically FreeBSD by another name. > > That's not an accident. As a DragonFly developer that mostly works on > userland, I've been converging FreeBSD and DragonFly again. They > drifted apart after 10 years, but I thought it better to keep the > userlands as similar as possible for 3rd party applications. > >> Yes, a ton of work has been done on the kernel, file systems, maybe ports. >> But the ABI is almost the same -- heck, for the most part, >> Linux==NetBSD==FreeBSD==OpenBSD. >> Sure, they might be implemented differently, but they are highly source >> compatible and every highly object code compatible. >>> I must be misunderstanding what you mean by "upgrade". It's the >>> cross-compiler that's complaining, and it was built by the latest patches. >> Agreed, we are both missing something simple. >> Usually this is the error you get when you don't upgrade >> Target.i3/Target.m3. >> If you want to press on with the current approach and ignore my advise, > > I'm not ignoring it; it's more like I'm not quite ready to give up since > I'm so close. I think I could actually get these libs and cm3 to build > on DragonFly if I used AMD64_FREEBSD as the target with the > cm3cg-AMD_DRAGONFLY generator. It was building before. I just figured > I'd then get stuck on the DragonFly side trying to compile cm3 with > itself, theoretically I should get the same error there. > > >> Ugh. I saw something about that in your diffs. Definitely we intend to >> link statically. >> I actually went to the trouble of removing use of gmp/mpfr/mpc in >> current source. >> They are used for actually very little and aren't worth it. > Hi Jay, If you get those patches committed and publish an official snapshot (compressed tarball) with those changes in it, then I'll update the FreeBSD port to that snapshot. If new patches are required, say to convert FreeBSD to use the C backend, I'll feed those back too. Then I'll try porting DragonFly again, your way. Thanks, John From jay.krell at cornell.edu Fri Jan 24 09:45:39 2014 From: jay.krell at cornell.edu (Jay K) Date: Fri, 24 Jan 2014 08:45:39 +0000 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: <52E1A8C9.3010903@marino.st> References: <52DBA2EF.4020101@marino.st>, , <52DBC081.1070401@marino.st> , <52DBC748.5090308@marino.st> <52DBCDBA.8030809@marino.st>,<52E1A8C9.3010903@marino.st> Message-ID: sorry, I'm delayed..so much to do.. I commited the networking fix.. The m3core/unix one I think we already have. Later... - Jay > Date: Fri, 24 Jan 2014 00:42:01 +0100 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) > > On 1/19/2014 14:06, John Marino wrote: > > On 1/19/2014 13:50, Jay K wrote: > >> And there are advantages not yet implemented, in particular, efficient > >> exception handling by generating C++. > > > > Like FreeBSD (and unlike NetBSD and OpenBSD), DragonFly has implemented > > dl_iterate_phdr which gcc uses for efficient zero-cost exception > > handling. That might come in handy here (I implemented it in > > DragonFly's real-time linker, mainly for Ada/GNAT but C++ benefited.) > > > > > >> Your patches should apply fairly cleanly to any new checkout. > >> I am willing to through them and apply them to CVS if you want -- > >> depends on if you are ok with my stealing your credit, vs. you want to > >> commit them yourselves. > > > > I would love it if you did this - I'm happy with a mention in the commit > > message. I just attached a tarball of all the patches I'm using right > > now. Some of these changes are specifically for the cross compiler (I'm > > think of the /scripts changes mostly) but most are valid assuming they > > still apply to the head of the repo. Please take as much as you want > > with my gratitude. > > > >> I already skimmed them and they all look easy. > >> DragonflyBSD is basically FreeBSD by another name. > > > > That's not an accident. As a DragonFly developer that mostly works on > > userland, I've been converging FreeBSD and DragonFly again. They > > drifted apart after 10 years, but I thought it better to keep the > > userlands as similar as possible for 3rd party applications. > > > >> Yes, a ton of work has been done on the kernel, file systems, maybe ports. > >> But the ABI is almost the same -- heck, for the most part, > >> Linux==NetBSD==FreeBSD==OpenBSD. > >> Sure, they might be implemented differently, but they are highly source > >> compatible and every highly object code compatible. > >>> I must be misunderstanding what you mean by "upgrade". It's the > >>> cross-compiler that's complaining, and it was built by the latest patches. > >> Agreed, we are both missing something simple. > >> Usually this is the error you get when you don't upgrade > >> Target.i3/Target.m3. > >> If you want to press on with the current approach and ignore my advise, > > > > I'm not ignoring it; it's more like I'm not quite ready to give up since > > I'm so close. I think I could actually get these libs and cm3 to build > > on DragonFly if I used AMD64_FREEBSD as the target with the > > cm3cg-AMD_DRAGONFLY generator. It was building before. I just figured > > I'd then get stuck on the DragonFly side trying to compile cm3 with > > itself, theoretically I should get the same error there. > > > > > >> Ugh. I saw something about that in your diffs. Definitely we intend to > >> link statically. > >> I actually went to the trouble of removing use of gmp/mpfr/mpc in > >> current source. > >> They are used for actually very little and aren't worth it. > > > > Hi Jay, > If you get those patches committed and publish an official snapshot > (compressed tarball) with those changes in it, then I'll update the > FreeBSD port to that snapshot. > > If new patches are required, say to convert FreeBSD to use the C > backend, I'll feed those back too. > > Then I'll try porting DragonFly again, your way. > Thanks, > John > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Fri Jan 24 10:11:02 2014 From: adacore at marino.st (John Marino) Date: Fri, 24 Jan 2014 10:11:02 +0100 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: References: <52DBA2EF.4020101@marino.st>, , <52DBC081.1070401@marino.st> , <52DBC748.5090308@marino.st> <52DBCDBA.8030809@marino.st>, <52E1A8C9.3010903@marino.st> Message-ID: <52E22E26.10402@marino.st> On 1/24/2014 09:45, Jay K wrote: > sorry, I'm delayed..so much to do.. > I commited the networking fix.. > The m3core/unix one I think we already have. > > Later... > - Jay Right, but there's no downloadable source snapshot. And if the DragonFly support isn't in there, I'll have to carry all those patches too. So my goal is to have a few patches as possible. Anyway, I have to base the port on something -- pulling latest SVN is rarely done, and CVS is never done. Some github projects don't make releases and needed to be pulled directly, but this requires first git to be built (lots of dependencies) and then github tweaks their service and everything fails. So I'm a big believer in static compressed tarballs. Which means I can't update FreeBSD cm3 version until one exists. theoretically I can package one myself, but it looks better if it's "official" and comes from one of the known m3 websites. Thanks, John From jay.krell at cornell.edu Fri Jan 24 10:59:24 2014 From: jay.krell at cornell.edu (Jay K) Date: Fri, 24 Jan 2014 09:59:24 +0000 Subject: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) In-Reply-To: <52E22E26.10402@marino.st> References: <52DBA2EF.4020101@marino.st>, , <52DBC081.1070401@marino.st> , <52DBC748.5090308@marino.st> <52DBCDBA.8030809@marino.st>,<52E1A8C9.3010903@marino.st> , <52E22E26.10402@marino.st> Message-ID: Please be patient. We'll get a snapshot, soon. A release, unclear. - Jay > Date: Fri, 24 Jan 2014 10:11:02 +0100 > From: adacore at marino.st > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Stuck on adding new cm3 target (AMD64_DRAGONFLY) > > On 1/24/2014 09:45, Jay K wrote: > > sorry, I'm delayed..so much to do.. > > I commited the networking fix.. > > The m3core/unix one I think we already have. > > > > Later... > > - Jay > > > Right, but there's no downloadable source snapshot. And if the > DragonFly support isn't in there, I'll have to carry all those patches > too. So my goal is to have a few patches as possible. > > Anyway, I have to base the port on something -- pulling latest SVN is > rarely done, and CVS is never done. Some github projects don't make > releases and needed to be pulled directly, but this requires first git > to be built (lots of dependencies) and then github tweaks their service > and everything fails. So I'm a big believer in static compressed > tarballs. > > Which means I can't update FreeBSD cm3 version until one exists. > theoretically I can package one myself, but it looks better if it's > "official" and comes from one of the known m3 websites. > > Thanks, > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Jan 24 11:38:01 2014 From: jay.krell at cornell.edu (Jay K) Date: Fri, 24 Jan 2014 10:38:01 +0000 Subject: [M3devel] garbage collector compacting and volatile? Message-ID: Is our garbage collector compacting? In particular, what model do we need for "volatile" in codegen? Does the compactor read the registers or just the stack? Update references anywhere? Just the stack? Also context? There are tradeoffs either way. If it is not compacting and only reads the stack, then it can be more portable, but every write of a collected type/pointer would have to be volatile. Or is that what the barriers are for? If we update values on the stack, then there is some portability, but reads and writes would have to be volatile. If we read and write registers in the collector, then nothing has to be volatile. Should this all be parameters passed to the backend? And/or preserved via #ifdef in the C backend? That last part is easy enough, for generated C to take #ifdefs indicating if reads and/or writes should be volatile. If the collector is not compacting, then it need not update stack or register values, just possibly read them. I recall reading that it is compacting. Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Fri Jan 24 19:24:35 2014 From: hosking at cs.purdue.edu (Tony Hosking) Date: Fri, 24 Jan 2014 13:24:35 -0500 Subject: [M3devel] proposal for getting wow64 context/stack reliably? In-Reply-To: References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL> Message-ID: As a way forward I propose that we indeed treat EXTERNAL calls as needing to capture a GC context. At least that should be the default. Some calls could be annotated as not needing to capture the GC context with a different EXTERNAL pragma. Knowing that such a call can safely ignore saving the context would require inspection of the external function, or understanding of its behavior. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Mobile +1 765 427 5484 On Jan 23, 2014, at 6:09 PM, Jay K wrote: > Yes. Agreed. > They should be ok with respect to SuspendThread + GetThreadContext. The small C++ test program in scratch. > I don't know about the "other" problem that affects even pthreads and I agree we should focus on that first, on pthreads, native I386_NT, and possibly AMD64_NT, but for now, at least briefly, ignore I386_NT on 64bit Windows. > > > - Jay > > > From: rcolebur at SCIRES.COM > To: m3devel at elegosoft.com > Date: Thu, 23 Jan 2014 22:23:43 +0000 > Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably? > > Jay: > > When you say native 32/64-bit ?should be ok?, you really mean ok with respect to the SYSWOW64 problem you?ve identified. > > Both of these native implementations still have threading misbehaviors as identified by the thread test program. Correct? > > --Randy > > From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K > Sent: Thursday, January 23, 2014 2:57 PM > To: Tony; Coleburn, Randy > Cc: m3devel > Subject: EXT:RE: [M3devel] EXT: proposal for getting wow64 context/stack reliably? > > There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same. > Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok. > > - Jay > From: hosking at cs.purdue.edu > Date: Thu, 23 Jan 2014 11:51:02 -0500 > To: rcolebur at SCIRES.COM > CC: m3devel at elegosoft.com; jay.krell at cornell.edu > Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack reliably? > > Yes, agreed. > We need to address the native thread problem on all platforms (both pthread and windows). > > However, this particular problem is pernicious if we need to save M3 stack state at all external calls. > > Are there any linker tricks for Windows one can use to wrap only system calls? > > I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped, This approach is used for all modern Java implementations that I am aware of. > > It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state. > > Antony Hosking | Associate Professor | Computer Science | Purdue University > 305 N. University Street | West Lafayette | IN 47907 | USA > Mobile +1 765 427 5484 > > > > > > > On Jan 23, 2014, at 6:59 AM, Coleburn, Randy wrote: > > Jay, > > If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. > > Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. > > Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. > > Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows? > > --Randy > > Sent from my iPhone > > On Jan 23, 2014, at 2:58 AM, "Jay K" wrote: > There is a behavior/bug in wow64. > Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc. > > > > SuspendThread / GetThreadContext work like this: > > > 32bit processes consist almost entirely of 32bit code. > There is a small amount of 64bit code. > > > If you suspend while running 32bit code, GetThreadContext works. > if you suspend while running 64bit code, GetThreadContext usually but not always works. > > > 64bit code is run en route to syscalls. > For example you call: > 1 kernel32!Sleep > 2 it calls 32bit NtDelayExecution > 3 that calls wow64NtDelayExecutation (via a cross segment "far" jmp or call) > 4 which calls native NtDelayExecuation > > > In between 2 and 3, within 64bit code, the 32bit context is saved. > You can step through it very easily in a debugger. Really. > Where GetThreadContext knows where to get it. > The problem is that saving context is not atomic. > You can suspend while saving context. > > > What to do? > > > scratch/wow64stack contains a program that detects the bug. > I believe it is the basis of a workaround for the bug. > > > Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, > not only functions that use exception handling, but also functions that call "extern" functions > call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection > use that, if it isn't zero. > Normally it will be zero. > Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile. > That is why it isn't a set/set-zero pattern (like in the test program). > > > Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. > > > Rejected counter proposal: > Don't suspend/gc when running a syscall. > No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that. > > > Possible augmentation: if running native, short circuit most of this > > > Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate > compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. > > > AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, > until/unless there is another 32bit platform runable on some similar 64bit platforms. > > > Performance impact: hypothetically large but probably not noticable. > > > Furthe refinements: It isn't extern/native code per se, it is syscalls. > We could further augment pragmas to discern them. > > > We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls > to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. > It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... > > > > Agreed? I'll make the compiler change? > > > Oh, also, not just stack pointer, but other registers, at least non-volatiles? > > > Eventually cooperative suspend will cause this to fall away as a problem.. > > > - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Fri Jan 24 19:12:33 2014 From: hosking at cs.purdue.edu (Tony Hosking) Date: Fri, 24 Jan 2014 13:12:33 -0500 Subject: [M3devel] garbage collector compacting and volatile? In-Reply-To: References: Message-ID: <8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu> On Jan 24, 2014, at 5:38 AM, Jay K wrote: > Is our garbage collector compacting? Yes, it is mostly-copying. Only pages that are not referred to from the stack (i.e., only from other objects) have their objects relocated (compacted). > In particular, what model do we need for "volatile" in codegen? > Does the compactor read the registers or just the stack? It reads both registers and stack. > Update references anywhere? Just the stack? Also context? The collector first pins all pages referenced from the stacks, including registers. Any page not pinned has its reachable objects evacuated and all references to those objects (i.e., from other objects) are updated to refer to the the new location. > There are tradeoffs either way. Indeed. Too many to enumerate here. > If it is not compacting and only reads the stack, then it can be more > portable, but every write of a collected type/pointer would have to be volatile. > Or is that what the barriers are for? The write barrier allows the collector to run concurrently with the mutator (once all stack roots have been sampled). > If we update values on the stack, then there is some portability, > but reads and writes would have to be volatile. No updates to values on the stacks. > If we read and write registers in the collector, then nothing has to be volatile. Read only. No writes to registers. > Should this all be parameters passed to the backend? Nope. > And/or preserved via #ifdef in the C backend? > That last part is easy enough, for generated C to take #ifdefs > indicating if reads and/or writes should be volatile. Nope. > If the collector is not compacting, then it need not update > stack or register values, just possibly read them. > > I recall reading that it is compacting. It is mostly-copying. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sat Jan 25 05:04:59 2014 From: jay.krell at cornell.edu (Jay K) Date: Sat, 25 Jan 2014 04:04:59 +0000 Subject: [M3devel] proposal for getting wow64 context/stack reliably? In-Reply-To: References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>, , Message-ID: I might yet have another solution here..there might be a way to detect the race and retry...later.. - Jay From: hosking at cs.purdue.edu Date: Fri, 24 Jan 2014 13:24:35 -0500 To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably? As a way forward I propose that we indeed treat EXTERNAL calls as needing to capture a GC context. At least that should be the default.Some calls could be annotated as not needing to capture the GC context with a different EXTERNAL pragma. Knowing that such a call can safely ignore saving the context would require inspection of the external function, or understanding of its behavior. Antony Hosking | Associate Professor | Computer Science | Purdue University305 N. University Street | West Lafayette | IN 47907 | USAMobile +1 765 427 5484 On Jan 23, 2014, at 6:09 PM, Jay K wrote:Yes. Agreed. They should be ok with respect to SuspendThread + GetThreadContext. The small C++ test program in scratch. I don't know about the "other" problem that affects even pthreads and I agree we should focus on that first, on pthreads, native I386_NT, and possibly AMD64_NT, but for now, at least briefly, ignore I386_NT on 64bit Windows. - Jay From: rcolebur at SCIRES.COM To: m3devel at elegosoft.com Date: Thu, 23 Jan 2014 22:23:43 +0000 Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably? Jay: When you say native 32/64-bit ?should be ok?, you really mean ok with respect to the SYSWOW64 problem you?ve identified. Both of these native implementations still have threading misbehaviors as identified by the thread test program. Correct? --Randy From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Thursday, January 23, 2014 2:57 PM To: Tony; Coleburn, Randy Cc: m3devel Subject: EXT:RE: [M3devel] EXT: proposal for getting wow64 context/stack reliably? There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same. Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok. - JayFrom: hosking at cs.purdue.edu Date: Thu, 23 Jan 2014 11:51:02 -0500 To: rcolebur at SCIRES.COM CC: m3devel at elegosoft.com; jay.krell at cornell.edu Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack reliably? Yes, agreed.We need to address the native thread problem on all platforms (both pthread and windows). However, this particular problem is pernicious if we need to save M3 stack state at all external calls. Are there any linker tricks for Windows one can use to wrap only system calls? I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped, This approach is used for all modern Java implementations that I am aware of. It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state. Antony Hosking | Associate Professor | Computer Science | Purdue University305 N. University Street | West Lafayette | IN 47907 | USAMobile +1 765 427 5484 On Jan 23, 2014, at 6:59 AM, Coleburn, Randy wrote: Jay, If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows? --Randy Sent from my iPhone On Jan 23, 2014, at 2:58 AM, "Jay K" wrote: There is a behavior/bug in wow64. Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc. SuspendThread / GetThreadContext work like this: 32bit processes consist almost entirely of 32bit code. There is a small amount of 64bit code. If you suspend while running 32bit code, GetThreadContext works. if you suspend while running 64bit code, GetThreadContext usually but not always works. 64bit code is run en route to syscalls. For example you call: 1 kernel32!Sleep 2 it calls 32bit NtDelayExecution 3 that calls wow64NtDelayExecutation (via a cross segment "far" jmp or call) 4 which calls native NtDelayExecuation In between 2 and 3, within 64bit code, the 32bit context is saved. You can step through it very easily in a debugger. Really. Where GetThreadContext knows where to get it. The problem is that saving context is not atomic. You can suspend while saving context. What to do? scratch/wow64stack contains a program that detects the bug. I believe it is the basis of a workaround for the bug. Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms, not only functions that use exception handling, but also functions that call "extern" functions call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection use that, if it isn't zero. Normally it will be zero. Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile. That is why it isn't a set/set-zero pattern (like in the test program). Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling. Rejected counter proposal: Don't suspend/gc when running a syscall. No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that. Possible augmentation: if running native, short circuit most of this Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets. AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT, until/unless there is another 32bit platform runable on some similar 64bit platforms. Performance impact: hypothetically large but probably not noticable. Furthe refinements: It isn't extern/native code per se, it is syscalls. We could further augment pragmas to discern them. We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls to any functions that make syscalls. This is likely too hard to maintain and too unfriendly. It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc... Agreed? I'll make the compiler change? Oh, also, not just stack pointer, but other registers, at least non-volatiles? Eventually cooperative suspend will cause this to fall away as a problem.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Sat Jan 25 05:38:03 2014 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Fri, 24 Jan 2014 23:38:03 -0500 Subject: [M3devel] proposal for getting wow64 context/stack reliably? In-Reply-To: References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL> Message-ID: <20140125043803.GA3423@topoi.pooq.com> On Sat, Jan 25, 2014 at 04:04:59AM +0000, Jay K wrote: > I might yet have another solution here..there might be a way to > detect the race and retry...later.. > > - Jay I seem to remember a lot of comments in the trestle code tht seem to be intended as assertions to catch race conditions. Is any of that stuff conceivably helpful here? -- hendrik From jay.krell at cornell.edu Sun Jan 26 07:08:20 2014 From: jay.krell at cornell.edu (Jay K) Date: Sun, 26 Jan 2014 06:08:20 +0000 Subject: [M3devel] proposal for getting wow64 context/stack reliably? In-Reply-To: <20140125043803.GA3423@topoi.pooq.com> References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>, , , , <20140125043803.GA3423@topoi.pooq.com> Message-ID: No. > Date: Fri, 24 Jan 2014 23:38:03 -0500 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably? > > On Sat, Jan 25, 2014 at 04:04:59AM +0000, Jay K wrote: > > I might yet have another solution here..there might be a way to > > detect the race and retry...later.. > > > > - Jay > > I seem to remember a lot of comments in the trestle code tht seem to > be intended as assertions to catch race conditions. > Is any of that stuff conceivably helpful here? > > -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Jan 27 08:23:26 2014 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jan 2014 07:23:26 +0000 Subject: [M3devel] proposal for getting wow64 context/stack reliably? In-Reply-To: References: <0BB8FA59C2932741A3A2941A8B9D8BFF9252DCC5@ATLEX04-SRV.SCIRES.LOCAL>, , , , , , , , <20140125043803.GA3423@topoi.pooq.com>, Message-ID: > I might yet have another solution here And it might only be for Windows 8 or newer.I tend to think we should go with the earlier proposal -- save context ourselves in thread locals when calling externals. Likely portable implementation is setjmp or getcontext, though we can maybe do better.This would be part of cooperative suspend anyway, which we want to move to. - Jay From: jay.krell at cornell.edu To: hendrik at topoi.pooq.com; m3devel at elegosoft.com Date: Sun, 26 Jan 2014 06:08:20 +0000 Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably? No. > Date: Fri, 24 Jan 2014 23:38:03 -0500 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably? > > On Sat, Jan 25, 2014 at 04:04:59AM +0000, Jay K wrote: > > I might yet have another solution here..there might be a way to > > detect the race and retry...later.. > > > > - Jay > > I seem to remember a lot of comments in the trestle code tht seem to > be intended as assertions to catch race conditions. > Is any of that stuff conceivably helpful here? > > -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Jan 27 08:45:08 2014 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jan 2014 07:45:08 +0000 Subject: [M3devel] garbage collector compacting and volatile? In-Reply-To: <8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu> References: , <8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu> Message-ID: In summary, shortest: We must be able to read registers (or save them to the stack or elsewhere (known thread locals)). We must be able to read the stack. We need not write registers or stack (from garbage collector). Slight elaboration: Copying/compaction only occurs when references are only from globals and heap. Therefore updates/writes only occur to globals and heap. Is there any desire to do better/different? To require the ability to update stack/registers? What does Java typically to? I guess there is the problem that we don't know for certain if values in registers and stack are pointers, or just coincident integers, and so updating them can be wrong.Whereas in the globals and "reachable thereof" we have precise type information for, so we can update those values safely.To do better/different would require more backend cooperation.We conservatively treat registers/stack as pointers if they look like them, and therefore keep values alive, but not trust them so much as to update them. Thanks, - Jay From: hosking at cs.purdue.edu Date: Fri, 24 Jan 2014 13:12:33 -0500 To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] garbage collector compacting and volatile? On Jan 24, 2014, at 5:38 AM, Jay K wrote: Is our garbage collector compacting? Yes, it is mostly-copying.Only pages that are not referred to from the stack (i.e., only from other objects) have their objects relocated (compacted). In particular, what model do we need for "volatile" in codegen? Does the compactor read the registers or just the stack? It reads both registers and stack. Update references anywhere? Just the stack? Also context? The collector first pins all pages referenced from the stacks, including registers.Any page not pinned has its reachable objects evacuated and all references to those objects (i.e., from other objects) are updated to refer to the the new location. There are tradeoffs either way. Indeed. Too many to enumerate here. If it is not compacting and only reads the stack, then it can be more portable, but every write of a collected type/pointer would have to be volatile. Or is that what the barriers are for? The write barrier allows the collector to run concurrently with the mutator (once all stack roots have been sampled). If we update values on the stack, then there is some portability, but reads and writes would have to be volatile. No updates to values on the stacks. If we read and write registers in the collector, then nothing has to be volatile. Read only. No writes to registers. Should this all be parameters passed to the backend? Nope. And/or preserved via #ifdef in the C backend? That last part is easy enough, for generated C to take #ifdefs indicating if reads and/or writes should be volatile. Nope. If the collector is not compacting, then it need not update stack or register values, just possibly read them. I recall reading that it is compacting. It is mostly-copying. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Mon Jan 27 16:16:57 2014 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 27 Jan 2014 10:16:57 -0500 Subject: [M3devel] garbage collector compacting and volatile? In-Reply-To: References: , <8C35EFAF-82C8-4757-B685-61C013ED4BA3@cs.purdue.edu> Message-ID: <4798B41F-079F-463C-A4FB-C36185B29997@cs.purdue.edu> On Jan 27, 2014, at 2:45 AM, Jay K wrote: > In summary, shortest: > > > We must be able to read registers (or save them to the stack or elsewhere (known thread locals)). > We must be able to read the stack. > We need not write registers or stack (from garbage collector). Correct. > Slight elaboration: > Copying/compaction only occurs when references are only from globals and heap. > Therefore updates/writes only occur to globals and heap. Correct. > Is there any desire to do better/different? > To require the ability to update stack/registers? Without control of the register allocator, optimisations, and stack layout performed by the compiler (especially back-end) getting fully accurate stack maps is very difficult. > What does Java typically to? Java VMs usually control their own destiny, having full control of JIT and stack layout. viz. HotSpot, IBM J9, Jikes RVM. > I guess there is the problem that we don't know for certain if values in registers and stack are pointers, or just coincident integers, and so updating them can be wrong. Correct. > Whereas in the globals and "reachable thereof" we have precise type information for, so we can update those values safely. Correct. > To do better/different would require more backend cooperation. > We conservatively treat registers/stack as pointers if they look like them, and therefore keep values alive, but not trust them so much as to update them. Correct. > > > Thanks, > - Jay > > From: hosking at cs.purdue.edu > Date: Fri, 24 Jan 2014 13:12:33 -0500 > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] garbage collector compacting and volatile? > > On Jan 24, 2014, at 5:38 AM, Jay K wrote: > > Is our garbage collector compacting? > > Yes, it is mostly-copying. > Only pages that are not referred to from the stack (i.e., only from other objects) have their objects relocated (compacted). > > In particular, what model do we need for "volatile" in codegen? > Does the compactor read the registers or just the stack? > > It reads both registers and stack. > > Update references anywhere? Just the stack? Also context? > > The collector first pins all pages referenced from the stacks, including registers. > Any page not pinned has its reachable objects evacuated and all references to those objects (i.e., from other objects) are updated to refer to the the new location. > > There are tradeoffs either way. > > Indeed. Too many to enumerate here. > > If it is not compacting and only reads the stack, then it can be more > portable, but every write of a collected type/pointer would have to be volatile. > Or is that what the barriers are for? > > The write barrier allows the collector to run concurrently with the mutator (once all stack roots have been sampled). > > If we update values on the stack, then there is some portability, > but reads and writes would have to be volatile. > > No updates to values on the stacks. > > If we read and write registers in the collector, then nothing has to be volatile. > > Read only. No writes to registers. > > Should this all be parameters passed to the backend? > > Nope. > > And/or preserved via #ifdef in the C backend? > That last part is easy enough, for generated C to take #ifdefs > indicating if reads and/or writes should be volatile. > > Nope. > > If the collector is not compacting, then it need not update > stack or register values, just possibly read them. > > I recall reading that it is compacting. > > It is mostly-copying. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Mon Jan 27 20:46:23 2014 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 27 Jan 2014 13:46:23 -0600 Subject: [M3devel] New compile failure Message-ID: <52E6B78F.7000003@lcwb.coop> The compiler in the head, with some recent changes, gives: rodney at yellowstone:~/proj/m3/cm3-head/cm3/m3-sys/m3middle$ cm3 Fatal Error: configuration file didn't specify BUILD_DIR This message is coming from cm3/src/Main.m3, which has recently changed, but there are related changes to other source files too. Reverting Main.m3 to the previous version won't compile. Putting an older executable for cm3 into the bin directory, with recently updated and installed cm3.cfg and config works. From rodney_bates at lcwb.coop Mon Jan 27 21:49:43 2014 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 27 Jan 2014 14:49:43 -0600 Subject: [M3devel] narrow still failing.. In-Reply-To: <52E13487.9090504@lcwb.coop> References: <20140119213649.403AE1A208F@async.async.caltech.edu> <20140119220123.0D1971A208F@async.async.caltech.edu> <52E0202F.3020307@lcwb.coop> <20140122204749.C724E1A208F@async.async.caltech.edu> <52E13487.9090504@lcwb.coop> Message-ID: <52E6C667.6030708@lcwb.coop> It should be fixed in the head, now. On 01/23/2014 09:25 AM, Rodney M. Bates wrote: > > > On 01/22/2014 02:47 PM, mika at async.caltech.edu wrote: >> Aha so declaring the type as <: ROOT will probably work as a workaround? >> > > I'm not sure whether that would do it or not. In my pared-down case, I had > moved BDD.T and BDD.Root into main, but left the REFANY variables. That made > the failures go away. > > Until it is fixed, I think the best workaround is TYPECASE with a binding: > > TYPECASE x > OF BDD.T(b) => > Format(b), etc. >